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Micresoft 


Where do you want to go today? 


What do you get when you cross intense industry scrutiny with a burning desire to prove yourself? 
On this occasion, something totally unprecedented. With the latest editions of the Microsoft® Visual Studio® 
development system and Microsoft SQL Server™ 7.0, we believe that Microsoft can now offer customers a 
viable enterprise development platform. And all things considered, we wouldn’t dare make such a bold statement 
if we didn’t believe it ourselves. Why are we so Sure? For starters we’ve spent the last six years doggedly 
questioning enterprise developers about the tools they require to do their jobs. And everything we learned | 
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REGULAR EXPRESSIONS 19 
by Brian W. Kernighan and Rob Pike 

Regular expressions, one of the most broadly applicable of programmer’s tools, provide a compact 

and expressive notation for describing patterns of text. They are also algorithmically interesting, 

easy to implement, and highly useful. 


DESIGNING ALGORITHMS INCREMENTALLY 26 
by Udi Manber 

The incremental approach to addressing algorithmic problems Udi presents here is particularly 

useful as a way of arriving at possible new algorithms. 


THREAD COMMUNICATION IN PARALLEL ALGORITHMS 32 
by Lalit Pant 

With the increasing availability of multiprocessing hardware, thread-based parallel algorithms are 
becoming more and more important. Lalit presents thread communication mechanisms for use 

within parallel algorithms. hie 


SIMULATED RECURSION 40 

by Earl Augusta 

Recursion is a powerful tool for attacking problems where questions are repeated and the same 
actions are performed. C and Pascal allow recursion, but languages like COBOL generally forbid it. 


THE HMAC ALGORITHM 46 
by William Stallings 

The Message Authentication Code (MAC) is a widely used technique for performing message 
authentication. HMAC (short for “keyed-Hashing for Message Authentication”), a variation on the 

MAC algorithm, has emerged as an Internet standard for a variety of applications. 


THE PALMPILOT’S INFRARED PORT 32 | 
by A.J. Musgrove . | 
One of the powerful Palm III features is its support of data exchange over InfraRed (IR) ports. AJ. ene) Brae | 
explores programming the IR port by presenting both an IR test application and a version of the 
venerable game BattleShip that can be played between two players via the IR port. 
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EMBEDDED SYSTEMS | 


PARALLEL FUNCTIONAL DECISION TREES 62 | 
by Rene Schaad | 
Rene presents an approach to programming reactive situated agents that’s based on parallel 
functional decision trees. In the process, he introduces “InSitu,” a C++ class library and 
run-time system he’s developed and tested on mobile robots. 


INTERNET PROGRAMMING 


EXAMINING PERLDAP 2 
by Troy Neeriemer 

Netscape’s PerLDAP is an important tool for both programmers and administrators because 

it provides a mechanism for accessing directory information from Perl. Troy presents a high- 

level overview of PerLDAP, along with details of how you can use it. 
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Real-world data management solutions 
are typically more complex when one 
examines the pieces, than initially 
recognized by the majority of database 
programers. All software projects are 
complex puzzles comprised of many 


details, most of which are data-related. 


Often today’s “DBMS” solutions sacrifice 
the speed or control essential for a 


competitive application. 


c-tree Plus®, by FairCom, has been the 
choice of commercial developers for twenty 
years precisely because it offers the 
flexibility and control at the detail level to fit 
a wide variety of data management needs. 
Proven on large Unix servers and 


workstations, c-tree Plus’s small footprint 
and exceptional performance have also made 


it the engine of choice for professional 


developers on Windows and Mac. c-tree Plus 
offers sophisticated ISAM level control with 
which the developer may define precise data 
management solutions, making it a perfect 
fit for any development project requiring 
specific data handling features. 


c-tree Plus® offers the most 


mature ISAM solution today.. 


FairCom’s 
c-tree Plus 
database 
engine: 


° Advanced Indexing 
Technology 

¢« Complete Source Code 

° Complete Transaction 
Processing 

¢ ODBC Interface 

¢ Over 25 Developer’s 
Servers Included 

° Portable Multi-threaded 
API 

¢ Royalty Free 

e Standalone, Multi-user or 
Client/Server Models 

¢ Thread-Safe Libraries 

¢ Y2K Compliant 





Don’t wait, see for yourself! 
USA. 800.234.8180 = 
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FairGe 
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The FairCom 
Server: 


A solid, high performance 
database server that is 
scalable, portable and offers 
unequalled control. FairCom 
has been providing database 
solutions to the commercial 
development community for 
twenty years. You won’t find 
a better solution, with these 
features and performance 
anywhere else! 

° Client Side Source Code 
° File Encryption 

° File Mirroring Logic 

° Full Conditional 

Index Support 
° Full Heterogeneous 

Networking 
° Multiple Communication 

Protocols 
e Online Data Backup 
¢ Small Memory Footprint 
° Flexible OEM 

Licensing Options 
e Source Code Availability 











All these 
platforms 
supported in one 
package: 


MIPS ABI, DEC Alpha, Sun 
SPARC, Windows 9X, SCO, 
S8OPEN, AIX, RS/6000, 
HP9000, Sun OS, 
Interactive Unix, Linux 
(Alpha...), AT&T System V, 
QNX, Free BSD, OS/2, 
Mac, Windows NT, 
Windows 3.1, DOS, 
Netware NLM, & ee 
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by Kirby W. Angell 

JPython is a freely available version of Python implemented in 100 percent pure 
Java. Since JPython is written in Java, it is easy to include the JPython packages 
in a Java application and use JPython as your application’s scripting engine. 
JPython also makes an excellent tool for prototyping Java applets that are 
embedded in web browsers. 


PORTING C++ CODE FROM NT TO UNIX 84 
by George F. Frazier 

When porting large C++ programs from Windows to UNIX, it makes sense to use 

a tool that implements the Windows API natively on the target system. George 

uses MainSoft’s MainWin XDE (eXtended Development Environment) 3.1 to port 

a major application from Windows NT to Solaris. 
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by Michael Swaine 
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by Al Stevens 

Before putting the “scroll” back in his scrolling editor, Al asks the questions 
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by Steve Ball and John Miller Crawford 

A Java applet may seem to be an independent program, but in one crucial 

aspect it is not—if you change a static field of a class used within an applet, 
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dealing with this particularity. 
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by Jon Bentley 

This month, Jon presents tools and techniques for analyzing the performance 

of algorithms. Next month, he examines how code-tuning techniques speed up 

the various algorithms. 
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by Dennis E. Shasha 

The plans for a modular spacestation are up in the air, until Dr. Ecco and crew 
introduce some down-to-earth solutions. 
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by Gregory V. Wilson and Steve Chartley 

Greg looks at Tom Armstrong’s The Active Template Library, Jim Coplien’s 
Multi-Paradigm DESIGN for C++, and Carlton Egremont’s Mr. Bunny’s Guide to 

Active X, while Steve Chartley tackles Patterns in Java, by Mark Grand. 


Dr. Dobb’s Journal, April 1999 





APRIL 1999 
VOLUME 24, ISSUE 4 





EDITORIAL 8 


by Jonathan Erickson 

LETTERS 10 
by you 

NEWS & VIEWS 16 
by the DDJ staff 

OF INTEREST 126 
by Eugene Eric Kim 

SWAINE’S FLAMES 128 
by Michael Swaine 


RESOURCE CENTER 


As a service to our readers, source code and 
related files, and author guidelines are available 
at http://www.ddj.com/. Source code is also 
available via anonymous FTP from ftp.ddj.com/ 
(199.125.85.76). Letters to the editor, article 
proposals/submissions, and inquiries can be sent 
to editors@ddj.com/, faxed to 650-358-9749, or 
mailed to Dr. Dobb’s Journal, 411 Borel Ave., 
Suite 100, San Mateo, CA 94402-3522. 

For subscription questions, change of address, 
and orders, call 800-456-1215 (U.S. or Canada). 
For all other countries, call 303-678-8475 or fax 
303-661-1181. E-mail subscription questions to 
ddj@neodata.com/ or write to Dr. Dobb’s Journal, 
P.O. Box 56188, Boulder, CO 80322-6188. 

Back issues may be purchased for $9.00 per 
copy (includes shipping and handling). For issue 
availability, send e-mail to orders@mfi.com/, fax 
to 785-841-2624, or call 800-444-4881 (U.S. and 
Canada) or 785-838-7500 (all other countries). 
Back issue orders must be prepaid. Please send 
payment to Dr. Dobb’s Journal, 1601 W. 23rd 
Street, Suite 200, Lawrence, KS 66046-2700. 

Individual back articles may be purchased 
electronically at http://www.ddj.com/ as ZIP 
archives. 


NEXT MONTH 


Database development is in the May 
spotlight. 


FOR INSPIRED DATABASE APPS: 





THE. 
YOU CAN MAKE YOUR OWN. 


ouldn't it be great to write your app once, then port it to other databases and deploy it—without 
changing a line of code? With DBTools.h++ 3.0, you can! You'll save time and money when you use these 
elegantly designed, tested, and proven C++ libraries instead of writing your own. Now, DBTools.h++ 3.0 gives 
you native access to the newest database versions from Oracle, Microsoft, Sybase, Informix, and IBM, 


plus general connectivity through ODBC. 


For mapping relational database structures to C++ objects quickly, DBTools.h++ 3.0 includes DBFactory, 
You'll appreciate the way DBFactory attends to the details of relational-to-object mapping, allowing you to 
focus on application-specific logic. And what’s underlying DBTools.h++ 3.0? Tools.h++, the C++ foundation 
class library that’s become a de facto industry standard. Join the thousands of developers worldwide who 


depend on Rogue Wave components with confidence! 


| DBTools.h++. The masterpiece you can make your own. 


Download your free DBTools.h++ technical report at: WWW.roguewave.com/ad/db 
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Rogue Wave: 800-487-3217 - Stingray: 800-924-4223 - D.A.CH.: +49-6103-59 34-0 - UK: +44-118-988 0224 
France: +33-1-4196 2626 - Italy: +39-02-3809 3288 - Rest of Europe: +31-20-301 26 26 


Rogue Wave and .h++ are registered trademarks, and DBTools.h++ and DBFactory are trademarks of Rogue Wave Software, Inc. 
All other trademarks are the property of their respective holders. 








EDITORIAL 





And as luck would have it, that’s exactly what I got. But not just any LEGOs. No sir-ee Bob, 
a handful of red and yellow bricks isn’t for me. What I got was LEGO’s version of the full 
Ro bo f monty— the LEGO Mindstorms Robotics Invention System, with more than 700 LEGO pieces, 
light and touch sensors, a programmable embedded computer, infrared transmitter, software, and 
enough motors, gears, and wheels to roll out a pretty big barrel. Gee, I can’t tell you what a thrill 
it is to be the envy of every 12-year-old on the block. 
LEGO Mindstorms, developed by LEGO and the Massachusetts Institute of Technology, lets 
you design, build, and program mobile robots using LEGO pieces. At the heart of the system is 
a 16-MHz Hitachi H8/3292 microcontroller with an 8-channel 10-bit A/D 
a converter, serial communications interface, timers, and multiple I/O ports. The 
SRUTEIAE © eee re seam = AA-battery powered Mindstorms computer, officially called the “RCX” (short for 
A> —_ — — “Robotic Command Explorer”), has 16 KB ROM, 32 KB RAM, and is packaged in a 
=< : ie og “LEGO-enabled” case so that you can attach standard (and not so standard) LEGO 
bricks and beams to it. The RCX (about the size of a standard calculator) stores 
several programs that can be selected and executed via a small keypad and LCD. 

Software development takes place in a host/target environment, whereby you 
build programs on a Windows 95 PC, then download them—via an infrared 
mpenceres transmitter attached to the PC serial port— to the RCX. Programs are created using 
Bein aso | a visual environment called “RCX Code,” which is loosely based on the LOGO 
MESIUDIETE: “ programming language. With RCX Code, you connect on-screen program blocks 
isles: = (instructions) to build programs. Program blocks are categorized as “commands,” 

u “sensor watchers,” “stack controllers,” and “my commands.” Commands let you 
turn on/off motors, specify direction, set speeds, and the like. Sensor watchers let 
you specify how the robot responds to touch and light (or temperature and 
rotation, if you have the appropriate add-on kits). Stack controllers let you identify 
the different ways robot parts (motors, wheels, and so on) are to run based on sensor input, timers, 
or whatever. Finally, “my commands” allows you to organize stacks into groups. 

Program blocks are added top to bottom; you edit programs by inserting or deleting new blocks in 
between existing ones. The figure above (courtesy of the Mindstorms CD-ROM) shows both the 
development environment and part of a program that controls a robot. In this case, the robot is called 
“Track Talker” and uses two motors (one for each wheel) and a light sensor. The robot is designed to 
locate colored objects in its path. It repeatedly beeps as it follows a trail, but goes silent when it 
encounters a particular color. In the figure, the commands are on the left, the program “trcktlk” is in 
green (A and C control the motors attached to the A and C output ports on the robot), and the light 
sensor is controlled by the blue sensor watchers on the right (the sensor is attached to the robot’s 
input port 2). 

Of course, RCX Code isn’t the only way to write Mindstorms programs. LEGO also sells an SDK 
that lets you program the RCX using tools such as Visual Basic. And since the RCX is built around 
the Hitachi H8 microcontroller, you have the entire suite of GNU C tools at your fingertips. 

A slew of freeware/shareware programs have also cropped up, including Markus Noga’s legOS 
operating system (which will be presented in a upcoming issue of DD/), Dave Baum’s RCX 
Command Center, Barry Ruffner’s RCX Creator, and the like. As you might expect, there are a 
number of Mindstorms web sites, most of which can be found through the official site at 
http://www.legomindstorms.com/. Likewise, information for building do-it-yourself sensors is 
available at various web sites. 

But Mindstorms is more than a toy. In fact, it has become a very serious, yet inexpensive, 
research tool. At Denmark’s University of Aarhus (http://www.daimi.au.dk/~mic 
/speciale/index.html), for instance, Claus Bjerre, Jes Hvoldal, and Michael Nielsen are building 
“adaptive pet robots” like a robot dog that is capable of intelligent interaction with its 
environment. More specifically, the robot understands spoken commands, determines the source 
of the sound, and responds to those commands— depending on its current “mood” (hunger, 
anger, stress, and so on) which, in turn, is controlled by genetically evolved neural networks. 

Right. These guys are building robotic Fidos, and I’m still trying to find a connector peg to stick 
into a 1x8 beam. Thank goodness there are plenty of LEGO experts up and down the block— or 


will be as soon as school lets out for the day. 


Jonathan Erickson 
editor-in-chief 
jerickson@ddj.com 


ome on, there’s nothing wrong with someone my age wanting LEGOs for a birthday present. 
Lego of My C 


pe 





8 Dr. Dobb’s Journal, April 1999 


The only fully 


Sy My» Mioricalt Viaual fe 
Hoe 2 Mine inet Vina 


SG Viewone 
Rik Chagas 
OS EMytereget 
Gy Corder 
ed Cia Rorintet 


SiMccetaiate 


toudit 


te 


: 
| 
| 
| 
; 
b 
| 
| 
EE 
af 


@ rO3ified. Gmanning tor rev 
==> Chaseiiier modified: Cipiiodal 


AS sae Fine Pee e 





AVATS Ee] VANS) Ok Oo 


Thee topes habe 
Sronveaush date 


eished} 
Dieta 


Fafa Fa 


Gu Hour. 
i Moti 
id) Bt vi fale Fins 





integrated modeling tool for Microsoft Visual ornnrs 


e Fully integrated with the Visual C++ 6.0 IDE 
e Reverse engineers ATL/MFC 


¢ Microsoft® Visual SourceSafe™ Integration 


¢ Instantly updates UML diagrams based on code changes 


¢ Model your software with UML Class, Use Case and 
sequence diagrams 


e Automatic Roundtrip Engineering 
¢ Shows relationships to MFC, ATL and Stingray class libraries 


800-924-4223 
919-461-0672 
sales@stingray.com 


www.STINGRAY.com 


All products and brand names are trademarks and/or registered trademarks of their respective holders. 





LETTERS 





3D Modeling 

Dear DDJ, 

Regarding the “News & Views” entitled 
“Speeding Up 3D Modeling” in the Jan- 
uary 1999 issue of DDJ: I am concerned 
about the completeness and accuracy of 
the data sent to the program. If there is 
one scanner from a fixed point gathering 
data on a real-world 3D object, then only 
the surface that the fixed point would see 
could be recorded. Everything behind that 
front surface cannot be recorded and, 
therefore, will not exist to the scanner. Are 
multiple scanners from different locations 
around the object being used? Is anything 
being done to scan the inside structure of 
the object? 

I am also concerned about the accura- 
cy of the data being received. The B2 
Stealth bomber was specifically designed 
to reflect any energy (light, sound, and so 
on) hitting its surface from several ran- 
dom directions. Thus a scanner trying to 
detect a B2 would receive a weak and 
diffracted B2 signal. Other objects found 
in nature have similar properties. Is any- 
thing being done to ensure the integrity 
and accurateness of the signal being re- 
ceived by the scanner? 

Lastly, regarding the software itself, does 
the program do any checking on the data 
it receives? Also, does the program just 
work with surfaces or can it also process 
true 3D interior structures? 

James H. Sylvester 

Yorba Linda, California 


DDJ replies: You raise some good points, 
James. You can find out more about the 
techniques by contacting Cyra Technolo- 
gies at http://www.cyra.com/. 


Bill’s Inferno 

Dear DDJ, 

When reading Al Stevens’ January 1999 
column, I could not help but laugh out 
loud at his woes with the D-Flat 2000 
classes. When I first started programming 
for Windows in 1989 (Windows/386 at 
the time), I too was quite stubborn. I did 
not want to learn the API; heck, there 
were something like 450 functions! But 
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when I couldn’t find a decent interface 
library to make my life easier, I was 
forced by circumstances to “hunker 
down” (whatever that means in pro- 
gramming terms) and get a grip on the 
Windows API. 

And interestingly enough, that was the 
worst thing to happen to me as a pro- 
grammer but paradoxically the best thing 
for my career ds a programmer. On the 
one hand, I bought into the awful state 
of Windows instruction hook, line, and 
sinker —I abandoned any sort of struc- 
tured design in favor of page after page 
of message-handling switch statements. 
But at the same time my higher program- 
ming faculties were dying on the vine, my 
career could not have been more assured. 
For as I have written before, one’s value 
as a programmer for Windows does not 
consist in one’s abilities as a programmer. 
Rather, one’s value consists in one’s ar- 
cane knowledge of the many tricks, traps, 
pitfalls, and outright bugs and mistakes in 
the Windows API. 

Now I'll confess that I just couldn’t keep 
up after Windows for Workgroups. The 
API quickly grew beyond its initial hor- 
rendous size to encompass the thousands 
of calls—more if you count all the vari- 
ous ActiveX property/method gadgetry 
now under the hood— that it is today. 
But I still never lack for work for a sim- 
ple reason: Lots of people were as stub- 
born as Al was. And this has left them ut- 
terly clueless beyond the confines of MFC, 
OWL, and the like. Welcome to the entire 
Windows API. Al, I suggest abandoning 
all hope at your earliest possible conve- 
nience. Or perhaps I should be more ex- 
plicit: Welcome to Hell. 

John B. Williston 

wconsult@pacificnet.net 


Al replies: John, many thanks for your en- 
joyable letter. Beware. You remind me of 
an old friend who decided, in the early 
1980s, to establish himself as the world’s 
reigning authority on the MS-DOS phe- 
nomenum called the “terminate and stay 
resident” program (TSR). He built an ASM 
library around this peculiar construct, wrote 
articles, spoke at conferences, gave semi- 
nars, and built a company based on this 
specialty. Eventually, when TSRs became 
extinct, he went broke and is, at last re- 
port, producing adult web sites for a liv- 
ing. I wouldn’t want that to happen to you. 


More On the Crouch-Ecblin Effect 
Dear DDJ, 

There has been more than enough “snake- 
oil” purveyed to a relatively unsophisti- 
cated and vulnerable target market in the 
course of this Y2K run up. In this partic- 
ular instance, principled engineering and 


business practices combined to correct 
the condition you report about the so- 
called “Crouch-Echlin Effect” described 
in your February 1999 “News and Views” 
column. 

I am a senior systems engineer with 
Compaq and have been a principal in- 
vestigator of the Crouch-Echlin Effect 
since it was first called to our attention. 
Our specific interest has been deter- 
mining 1. whether or not this supposed 
Y2K phenomenon actually affected any 
Compaq platforms; and 2. what, if any- 
thing, such an effect required Compaq 
to do to obviate difficulties for our cus- 
tomers. 

In conjunction with Intel and others, 
Compaq has conducted exhaustive tests 
that yielded, essentially, no evidence that 
this purported phenomenon affects any 
contemporary ISA PC platform, and most 
certainly none of Compaq’s. Further, Com- 
paq does not endorse or offer for sale any 
“software fix” purported to address this 
“effect.” 

It is true that an agency of Digital Equip- 
ment Corp., prior to the acquisition by 
Compaq, was involved with Crouch and 
Echlin in investigating this phenomenon 
and, for a short time, that entity did li- 
cense, from Echlin, the right to distribute, 
for itself, his “software fix.” However, once 
a comprehensive technical investigation 
was conducted within Compaq, we de- 
termined that it was inappropriate for 
Compaq to appear to endorse or distribute 
this product and this activity was prompt- 
ly terminated. 

Whatever the “root cause” of this phe- 
nomenon — still not conclusively deter- 
mined, at this writing— there is no evi- 
dence to support the proposition that it 
can or should be addressed in software. 
Any platform that displays anomalous or 
disorderly “behaviors” of the character and 
magnitude attributed to this or any simi- 
lar phenomenon should simply not be 
used. Applying some “software fix” in a 
vain or misguided attempt to ameliorate 
such a condition is absolutely the most 
inappropriate response. Advocating such 
a response would be quite unethical and 
contrary to the best interests of both the 
advocate and the target audience/market. 

Both Compaq and Intel have published 
detailed statements concerning the inves- 
tigation and significance of the Crouch- 
Echlin Effect. The Intel statement is par- 
ticularly useful in debunking most of the 
misinformation that has been bruited about 
on Usenet and the Web concerning this 
phenomenon. Readers can find relevant 
information at both http://www.com- 
paq.com/year2000/year2000-qa.html and 
http://www. intel.com/support/year2000/ 
under4.htm. 
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(continued from page 10) 

I am not an authorized spokesperson 
for Compaq on this or any other matter. 
I did, however, participate fully in the in- 
vestigation and disposition of this issue. 
My sole interest, as a DD/ subscriber and 
as an individual well informed about all 
aspects of this whole Y2K issue, is to 
counter the incorrect information about 
Compag that you caused to be published. 

Austin Fletcher 

Compaq Computer Corp. 

austin. fletcher@compag.com 


Myopic about File Names 

Dear DDJ, 

I thought that “A Java Applet Search En- 
gine,” by Tim Kientzle (DDJ, February, 
1999) was useful and interesting. The tex- 
tual comments and highlight that “Java is 
somewhat myopic about filenames,” how- 
ever, prompted me to respond with some 
information that seems not to be widely 
known. The question of filenames and 
URLs first came to my attention about a 
year ago on the Java Developer Con- 
nection and I answered it with an appli- 
cation developed on NT under JDK 1.1.3. 
I am sending along new applet code de- 
veloped under 1.1.7B on Windows 98. 
(The complete source code is available 
electronically; see “Resource Center,” page 


5.) The code has been tested with ap- 
pletviewer, HotJava, and Microsoft Inter- 
net Explorer 4.01. The applet has not been 
tested on other platforms, but should 
work, assuming a proper JVM imple- 
mentation. 

The first piece of information regards 
CodeBase and DocumentBase. For what- 
ever reason, getCodeBase() returns the 
applet path MINUS the applet name. gef- 
DocumentBase(), on the other hand, re- 
turns the entire path, including the docu- 
ment name. CodeBase can be used 
immediately. If DocumentBase is neces- 
sary, one answer is to substring the val- 
ue from its beginning to the last instance 
of the file separator character (using the 
system property “file.separator” or the File 
variable “separator”) and prepending that 
to the file name. 

Second, the File class has a method, 
getCanonicalPath(), which will return 
the system dependent form of the file 
path. After properly setting up the URL 
and using URL.getFile(), this method will 
return the full path properly formatted 
for the native system. In the example ap- 
plet, I was lazy and used CodeBase. For 
testing purposes, I created a directory 
named “data” off the application direc- 
tory and created a file named “test.txt,” 
so my entry was “data/test.txt.” After 





pressing OK, the applet shows informa- 
tion regarding the bases and path. As Tim 
noted, it is unlikely that the current di- 
rectory will be that of an applet running 
from a browser, therefore the complete 
path is normally needed for a file (as op- 
posed to URL) entry. 

Joe Sam Shirah 

jshirah@ibm.net 


Ficl Update 

Dear DDJ, 

Thanks to Randy Leberknight, I have an 
update to my article “Ficl: An Embeddable 
Extension Language Interpreter” (DD/, Jan- 
uary 1999). In the example illustrating 
Conditionals on page 74, the positive | zero 
case underflows the stack. A dup and a 
drop fixes the problem. 


: signum ( x -- sign ) 
dup 0< if drop -1 
else 

O= if 0 else 1 endif 
endif 


) 


John Sadler 
john_sadler@alum.mit.edu. 
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epressions 


iia ogorinns, and software 


Brian W. Kernighan and Rob Pike 


he right programming language can make all the difference 
in how easy it is to write a program. This is why a pro- 
grammer’s arsenal holds not only general-purpose languages 
like C and its relatives, but also programmable shells, script- 
ing languages, and lots of application-specific languages. 

The power of good notation reaches beyond traditional pro- 
gramming into specialized problem domains. HTML lets us cre- 
ate interactive documents, often using embedded programs in 
other languages such as JavaScript. PostScript expresses an en- 
tire document as a specialized program. Spreadsheets and word 
processors often include languages to evaluate expressions, ac- 
cess information, and control layout. 

Regular expressions are one of the most broadly applicable spe- 
cialized languages, a compact and expressive notation for describing 
patterns of text. Regular expressions are algorithmically interest- 
ing, easy to implement in their simpler forms, and very useful. 

Regular expressions come in several flavors. The so-called 
“wildcards” used in command-line processors or shells to match 
patterns of file names are a particularly simple example. Typi- 
cally, “*” is taken to mean “any string of characters,” so a com- 
mand like 





del *.exe 


uses a pattern “*.exe” that matches all files with names that con- 
tain any string followed by the literal string “.exe”. 

Regular expressions pervade UNIX, in editors, tools like grep, 
and scripting languages like Awk, Perl, and Tcl. Although the 
variations among different programs may suggest that regular 
expressions are an ad hoc mechanism, they are, in fact, a lan- 
guage in a strong technical sense— a formal grammar specifies 
their structure and a precise meaning can be attached to each 
utterance in the language. Furthermore, the right implementa- 
tion can run very fast; a combination of theory and engineering 
practice pays off handsomely. 


The Language of Regular Expressions 

A regular expression is a sequence of characters that defines 
a pattern. Most characters in the pattern simply match them- 
selves in a target string, so the regular expression “abc” match- 
es that sequence of three letters wherever it occurs in the tar- 
get. A few characters are used in patterns as metacharacters 


Brian and Rob are researchers at Lucent Technologies’s Bell Labs 


and can be contacted at bwk@bell-labs.com and rob@bell-labs.com, 
respectively. 
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to indicate repetition, grouping, or positioning. In POSIX regu- 
lar expressions, “A” stands for the beginning of a string and “$” 
for the end, so “Ax” matches an “x” only at the beginning of a 
string, “x$” matches an “x” only at the end, “Ax$” matches “x” 
only if it is the sole character of the string, and “A$” matches the 
empty string. 

The character “.” (a period) matches any character, so “x.y” match- 
es “xay,” “x2y,” and so on, but not “xy” or “xyxy.” The regular ex- 
pression “A.$” matches a string that contains any single character. 

A set of characters inside brackets “[]” matches any single one 
of the enclosed characters; for example, “[0123456789]” match- 
es a single digit. This pattern may be abbreviated “[0-—9].” 

These building blocks are combined with parentheses for 
grouping, “|” for alternatives, “*” for zero or more occurrences, 
“+” for one or more occurrences, and “?” for zero or one oc- 
currences. Finally, “\” is used as a prefix to quote a metachar- 
acter and turn off its special meaning. 

These can be combined into remarkably rich patterns. For ex- 
ample, “\.[0—9]+” matches a period followed by one or more dig- 
its; “[O—9]+\.2[(0 —9]*” matches one or more digits followed by an 
optional period and zero or more further digits; “(\+|—)” matches 
a plus or a minus (“\+” is a literal plus sign); and “[eE](\+ |—)?[0-9}+” 
matches an “e” or “E” followed by an optional sign and one or more 
digits. These are combined in the following pattern that matches 
floating-point numbers: 


(\+|—)?C0—-9]+\.210—9]* | \.L0-9]+)(leE]C\ + | 210-9] +)? 





Example 1: The function match determines whether a 


string matches a regular expression. 








Example 2: The recursive function matchhere does most 
of the work. 


Example 3: The function matchstar is called when the 
expression begins with a starred character. 
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A Regular Expression Search Function 

Some systems include a regular expression library, usually called 
“regex” or “regexp.” However, if this is not available, it’s easy to 
implement a modest subset of the full regular expression lan- 
guage. The regular expressions we present here make use of 
four metacharacters: “A,” “$,” “.,” and “*,” with “*” specifying 
zero or more occurrences of the preceding period or literal char- 
acter. This provides a large fraction of the power of general reg- 
ular expressions with a tiny fraction of the implementation com- 
plexity. We’ll use these functions to implement a small but 
eminently useful version of grep (available electronically; see 
“Resource Center,” page 5). 

In Example 1, the function match determines whether a string 
matches a regular expression. If the regular expression begins 
with “A” the text must begin with a match of the remainder of 
the expression. Otherwise, we walk along the text, using match- 
here to see if the text matches at each position in turn. As soon 
as we find a match, we’re done. Expressions that contain “*” 
can match the empty string (for example, “.*y” matches “y” 
among many other things), so we must call matchhere even if 
the text is empty. 

In Example 2, the recursive function matchhere does most of 
the work. If the regular expression is empty, we have reached 
the end and thus have found a match. If the regular expression 
ends with “$,” it matches only if the text is also at the end. If 
the regular expression begins with a period, it matches any char- 
acter. Otherwise, the expression begins with a plain character 
that matches itself in the text. A “A” or “$” that appears in the 
middle of a regular expression is thus taken as a literal charac- 
ter, not a metacharacter. 

Notice that matchhere calls itself after matching one charac- 
ter of pattern and string. Thus the depth of recursion can be as 
much as the length of the pattern. 

The one tricky case occurs when the expression begins with 
a starred character, “x*,” for example. Then we call matchstar 
with three arguments — the operand of the star (x), the pattern 
after the star, and the text; see Example 3. Again, a starred reg- 
ular expression can match zero characters. The loop checks 
whether the text matches the remaining expression, trying at 
each position of the text as long as the first character matches 
the operand of the star. 

Our implementation is admittedly unsophisticated, but it works. 
And, at fewer than 30 lines of code, it shows that regular ex- 
pressions don’t need advanced techniques to be put to use. 


Grep 

The pattern-matching program grep, invented by Ken Thomp- 
son (the father of UNIX), is a marvelous example of the val- 
ue of notation. It applies a regular expression to each line of 
its input files and prints those lines that contain matching strings. 
This simple specification, plus the power of regular expres- 
sions, lets it solve many day-to-day tasks. In the following ex- 
amples, note that the regular expression used as the argument 
to grep is different from the wildcard pattern used to specify 
file names. 


e Search for a name in a set of source files: grep fprintf *.c 

e Search for a phrase in a set of text files: grep ‘regular expres- 
sion' * txt 

¢ Filter output from some other program, for example to print 
all error messages: gcc *.c | grep Error: 

e Filter input to some other program, for example to count non- 
blank lines: grep . *.cpp | wordcount 


With flags to print line numbers of matched lines, count match- 


es, do case-insensitive matching, select lines that don’t match 
the pattern, and other variations of the basic idea, grep is so 
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widely used that it has become the classic example of tool-based 
programming. 

Example 4 is the main routine of an implementation of grep 
that uses match. It is conventional that UNIX programs return 
0 for success and nonzero values for various failures. Our grep, 
like the UNIX version, defines success as finding a matching 
line, so it returns 0 if there were any matches, 1 if there were 
none, and 2 if an error occurred. These status values can be 
tested by other programs like a shell. 

As Example 5 illustrates, the function grep scans a single file, 
calling match on each line. This is mostly straightforward, but 
there are a couple of subtleties. First, the main routine doesn’t 
quit if it fails to open a file. This is because it’s common to say 
something like 


% grep herpolhode *.* 


and find that one of the files in the directory can’t be read. It’s 
better for grep to keep going after reporting the problem, rather 
than to give up and force users to type the file list manually to 
avoid the problem file. Second, grep prints the matching line 
and the file name, but suppresses the file name if it is reading 
standard input or a single file. This may seem an odd design, 
but it reflects a style of use based on experience. When given 
only one input, grep’s task is usually selection, and the file name 
would clutter the output. But if it is asked to search through 
many files, the task is most often to find all occurrences of some- 
thing, and the file names are helpful. Compare 


% strings enormous.dll | grep Error: 
with 


% grep grammer *.txt 


Our implementation of match returns as soon as it finds a 
match. For grep, that is a fine default. But, for implementing a 
substitution (search-and-replace) operator in a text editor, the 
leftmost longest match is more useful. For example, given the 





Example 4: Main routine of an implementation of grep 
that uses match. (Assumes command interpreter expands 
wild cards in file specification on the command line into 
lists of file names. UNIX shells exhibit this behavior, 
although MS-DOS COMMAND.COM does not.) 
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Example 5: The function grep scans a single file, calling 
match on each line. 


text “aaaaa,” the pattern “a*” matches the null string at the be- 
ginning of the text, but the user probably intended to match 
all five characters. To cause match to find the leftmost longest 
string, matchstar must be rewritten to be greedy: Rather than 
looking at each character of the text from left to right, it should 
skip over the longest string that matches the starred operand, 
then back up if the rest of the string doesn’t match the rest of 
the pattern. In other words, it should run from right to left. Ex- 
ample 6 is a version of matchstar that does leftmost longest 
matching. This might be the wrong version of matchstar for 
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Example 6: A version of matchstar that Hoes lenis ; 
longest matching. 





grep, because it does extra work; but for a substitution opera- 
tor, it is essential. 


What Next? 

Our grep is competitive with system-supplied versions, re- 
gardless of the regular expression. For example, it takes about 
six seconds to search a 40-MB text file on a 400-MHz Pentium 
(compiled with Visual C++). Pathological expressions can cause 
exponential behavior, such as “a*a*a*a*a*b” when given the 
input “aaaaaaaaac,” but the exponential behavior exists in 
many commercial implementations, too. A more sophisticat- 
ed matching algorithm can guarantee linear performance by 
avoiding backtracking when a partial match fails; the UNIX 
egrep program implements such an algorithm, as do scripting 
languages. 

Full regular expressions would include character classes like 
“[a-zA-Z]” to match a single alphabetic character; the ability to 
quote a metacharacter (for example, to search for a literal peri- 
od); parentheses like “(abc)*” for grouping; and alternatives, 
where “abc | def” matches “abc” or “def.” 

The first step is to help match by compiling the pattern into 
a representation that is easier to scan. It is expensive to parse a 
character class every time we compare it against a character; a 
precomputed representation based on bit vectors could make 
character classes much more efficient. 

For regular expressions with parentheses and alternatives, 
the implementation must be more sophisticated. One approach 
is to compile the regular expression into a parse tree that cap- 
tures its grammatical structure. This tree is then traversed to 
create a state machine — a table of states, each of which gives 
the next state for each possible input character. The string is 
scanned by the state machine, which reports when it reach- 
es a state corresponding to a match of the pattern. Another 
approach is similar to what is done in just-in-time compilers: 
The regular expression is compiled into instructions that will 
scan the string; the state machine is implicit in the generated 
instructions. 


Further Reading 

J.E.F. Friedl’s Mastering Regular Expressions (O’Reilly & As- 
sociates, 1997) is an extensive treatment of the subject. Reg- 
ular expressions are one of the most important features of 
some scripting languages; see The AWK Programming Lan- 
guage by A.V. Aho, B.W. Kernighan and PJ. Weinberger (Ad- 
dison-Wesley, 1988) and Programming Perl, by Larry Wall, 
Tom Christiansen, and Randal L. Schwartz (0 Reilly & Asso- 
ciates, 1996). 


DDJ 
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Udi Manber 


ur first intuition when looking at 

an algorithmic problem is to think 

of the whole input and try to find 

a solution for it. We carry with us 
the intuition of the physical world; when 
we try to solve a problem, we have to 
deal with all of it. The incremental 
method is different. It assumes that you 
have already found a solution, and now 
someone gives you a slightly larger prob- 
lem and asks you to extend the solution. 
If you can find a general way to extend 
the solution, then you can start with an 
empty input and keep extending it. Con- 
centrating on the incremental is often eas- 
ier and it leads to powerful techniques 
for attacking problems. 

I called this method the “induction 
method” in my book /ntroduction to Al- 
gorithms. A Creative Approach (Addison- 
Wesley, 1989), because it employs the 
same principle as mathematical induction. 
Since then, I found that induction intimi- 


Udi is a professor of computer science at 


the University of Arizona. He can be 
contacted at udi@cs.arizond.edu. 


26 





dates many people, and using it in this 
context can confuse things. So I'll stick 
with “incremental method” here, which is 
a clearer and more descriptive name. 
The maximum rectangle problem, de- 
scribed wonderfully by David Vandevoorde 


ies 


in his article “The Maximal Rectangle Prob- 
lem” (DD/J, April 1998), turns out to be a 
perfect example to illustrate the incremental 
method. (Vandevoorde referred to the prob- 
lem as the “maximal rectangle problem,” 
but “maximum” problem is more precise, 
because maximal usually means something 
that cannot be extended, whereas maxi- 
mum means the global maximum.) The in- 
cremental method is not perfect, and it is 
not always the best way, but it is applica- 
ble most of the time, and it is definitely a 
technique every algorithm designer should 
keep in their arsenal. 





The Maximum Rectangle Problem 
The maximum rectangle problem can be 
summed up as: 


The input: An NXM binary matrix B. 
The output: A rectangle in B of maximum 
area containing all 1s. 


The first issue when trying the incre- 
mental method is to define the incremen- 
tal. For matrices, there aren’t too many 
choices; either you add a new row or a 
new column. (1'll show later that this is 
not always straightforward.) Let’s try a new 
row. Assume that you have found the 
largest rectangle in the original matrix, and 


that you are now given one additional 


row. You need to determine if there is an 
even larger rectangle with the new row. 
That already limits your search. You need 
to consider only rectangles that end at the 
last row. So all the Os in the last row do 
not contribute. That means that you can 
divide the problem into several subprob- 
lems by looking only at intervals in the 
last row that contain only 1s; see Figure 
1. The subproblems are independent of 
one another, so you need to consider only 
one of them at a time. The same solution 
will apply to all of them. 

Looking at each subproblem, you are 
now looking for the maximum rectangle 
that ends at the last row, and you know 
that the last row contains all 1s. So you 
need to know, for each column 7, the 
length of the bottom block of 1s; call it 
C;. This kind of information is easy to 
maintain as you add rows; for each col- 
umn, you either increment the bottom 
block length Gf the last row contains a 1) 
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| oo (continued from page 206) 
 —Crt—“=ECR or set it to 0 Gif it contains a 0). The di- 


 ~=——r———SCT—CO—C. vision into subproblems is used here 
LL i mostly for simplicity; you can just as eas- 
— _ ily consider the whole row, with the 0s, 
as one problem. Given this information, 
you can now define your problem as a 
one-dimensional problem: 























The input: a sequence of integers, 
oe 

The output: the interval /i,j/ that maximizes 
the product of the length of the interval 
(j- i) times the minimum number in it. 




















(The area of a rectangle of all 1s corre- 
sponds to such a product; the width of 
the rectangle corresponds to the length of 
the interval and its height corresponds to 
the minimum number in it.) 

The incremental method lets you reduce 
a two-dimensional problem to a one- 
dimensional one. If you want to obtain an 
optimal OC(MN) solution for the original 
problem, you will need to solve this one- 
dimensional problem in linear time O(k). 
Using the incremental method again, you 
assume that you know how to solve the 
problem up to k numbers, and you now 
need to handle the k+1 number; call it 
i ae i i - Cys. If successful, this will reduce the 
Figure 1: Dividing the problem into subproblems and reducing it to a one- problem one more time and allow us to 
dimensional problem. concentrate on just one number. 
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You need to concentrate only on in- 
tervals that end at C,, and check how 
Cp; can change them to produce better 
intervals. If C,.,2C,, then the best inter- 
val that ends at C, can be extended to 
end at Cy.,, without changing its mini- 
mum number, so it is the best possible 
interval. But, as Figure 2 shows, if 
Ce+1<C, then you cannot easily deter- 
mine the best interval. Without C;,,=3, 
the best interval (2a) consists of the last 
two numbers, giving the product 
min{6,5}x2=10. Extending that interval to 
include the 3 (2b) gives min{6,5,3}x3=9, 
which is not better. But since the mini- 
mum number is now 3, you can use a 
bigger interval (2c), leading to a prod- 
uct of min{3,6,5,3}x4=12. If you extend 
it all the way (2d), you get a product of 
min{4,2,3,6,5,3}x6=12. There is no easy 
way to figure it out by looking at only 
Ces; and the best interval up to Cy. That 
seems like a failure of the incremental 
method, because concentrating on Cp. 
alone does not seem to be sufficient. 

But the incremental method can be ap- 
plied in many different ways, which is 
why it is so powerful. Incrementing by 
adding one number at a time to the end 
of the sequence is the natural thing to do, 
but it is not the only way to increment. 
Maybe instead of incrementing by look- 
ing at the last number, C;.,, you should 
look at the first number, C;? In this case, 
the first number has the exact same ef- 
fect as the last number, so this will not 
help. But what about the minimum num- 
ber in the sequence, or the maximum 
number? As it turns out, using the mini- 
mum number and using the maximum 
number lead to two different algorithms, 
both optimal. 

Start with the minimum number; call it 
m. For simplicity, Pll use m both to point 
to the number and to its value. How can 
m be involved in the solution? If the best 
interval contains m, then that interval 
might as well contain everything. In oth- 
er words, if m is involved then the best 
interval is the whole sequence. Keep that 
in mind, and check the alternatives. If m 
is not contained in the best interval, then 
the best interval is somewhere to the side 
of m, either to the left or to the right. You 
have now partitioned the problem into 
two independent subproblems, each of 
which can be solved recursively, so the 
problem is solved. In Figure 3, you see 
the intervals corresponding to choosing, 
from top to bottom, 2, 3, 4, 5, and 6. Get- 
ting a linear-time implementation of this 
solution is not straightforward, but since 
you spent only a constant time handling 
m, there is hope. 

Let’s try the incremental method con- 
centrating on the maximum number; call 
it M. This turns out to be slightly more 
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tricky, but it leads to an easier to imple- 
ment algorithm. Again, how can M be in- 
volved in the solution? There are two pos- 
sibilities when M is part of the interval. 
Either the interval consists of M by itself, 
in which case its area is Mx1 = M, or the 
interval includes one of M’s neighbors as 
well. In the latter case, M will contribute 
no more than its neighbor contributes. So 
if you can figure out which neighbor to 
choose, you can assign M’s contribution 
to that neighbor by specifying that the 
neighbor is worth twice its size, and do 
away with M. If you can do that safely, 
you have an incremental solution, because 
you can continue to handle maximum 
numbers one at a time. 

You allocate a counter to each number 
to indicate how many of its neighbors 
should be counted when this number is 
used for the best interval. You then re- 
move M and increment the counter of one 
of its neighbors (the mystery of which 
neighbor to choose will be revealed in a 
minute). In effect, what will happen is that 
when the minimum number in an inter- 
val is considered, all its larger neighbors 
will have been considered by then, and 
their number will be assigned. So at that 
point the smallest number will know the 
size of its interval. 

Suppose that the numbers are in de- 
creasing order with the largest on the left. 
The algorithm will take C, as the first 
rectangle by itself, then increment the 
counter of C, to 2. In the second step, 
C, counts twice (since its counter is 2), 
and so its rectangle is worth 2xC3. Check 
whether this is larger than the previous 
best rectangle (C,), and take the maxi- 
mum of them. You then assign C;,’s 
counter to C; which will be set at 3, and 
contribute 3XC; and so on. This is the 
simplest case, and in general the algo- 
rithm works just as well. 

Which neighbor to use— the smaller 
or the larger? The answer is the larger, 
because it is the safer choice. Any inter- 
val that contains one of M’s neighbors 
will at least contain M and the larger of 
the neighbors (it may contain both neigh- 
bors). So by assigning M’s contribution 
to the larger neighbor you ensure that M@ 
will be considered for any interval that 
contains it. An example of using this al- 
gorithm for the matrix in Figure 1 is in 
Figure 4. The contribution 6 is 1x6. The 
second largest is 5, whose counter is now 
2, so it contributes 2x5. Next is 4, which 
assigns its contributions to its neighbor 
2. Next is 3 Cd chose the 3 to the right, 
but you could just as well chose the oth- 
er one), which contributes 3x3. When the 
other 3 is considered, its counter is 4, so 
it contributes 4x3=12. The last one is 2, 
whose counter is now 6, leading to an- 
other best solution of 12. 
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The two algorithms suggested by the in- 
cremental method are not complete. You 
need to carefully design the right data struc- 
tures to handle all the operations efficiently. 
The devil is in the details. But the details 
are easier Once you have an overview and 
an understanding of the algorithm, which 
you now do. One of the strengths of the 


not crucial, because the main step of as- 
signing worth can be performed any time 
a number is larger than both its neighbors.) 


Conclusion 

The incremental method is particularly 
useful as a way to arrive at possible new 
algorithms. It can lead the way and sug- 





gest different approaches. It is not a 
magic bullet, and it will not replace in- 
tuition and experience, but it helps to 
develop both. 
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incremental method is the ease in which 
one gets to an algorithmic idea. 

The algorithm demonstrated in Figure 
4 consists of the following four steps, 
which are repeated for each number: 


1. Find the next maximum /. 

2. Check M’s current contribution against 
the previous best. 

3. Assign M’s counter to its larger neighbor. 

4. Delete M. 


To get a repeated list of maximums you 
need to sort the numbers. Fortunately, all 
the numbers are in the range of 1 to N 
(since they sum the number of 1s in a col- 
umn), so you can sort all of them in linear 
time using a simple bucket sort. Finding the 
neighbors of each number can be achieved 
by linking all the numbers in a doubly 
linked list, which also lets you delete each 
number. All these operations can be done 
in linear time. (The algorithm can be fur- 
ther simplified by noticing that the sort is 





Figure 4: Assigning the maximum to the larger neighbor. 
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Lalit Pant 


ith the increasing availability of 
symmetric multiprocessing (SMP) 
hardware, parallel algorithms are 
playing a vital role in extracting 
faster performance from computer sys- 
tems. But while the idea of parallelizing 
algorithms to achieve greater speeds is ap- 
pealing, there are practical difficulties that 
need to be overcome before this can be 
done effectively. Chief amongst them is 
the issue of efficient interthread commu- 
nication. Most parallel algorithms require 
some means by which threads can talk to 
each other. An inefficient communication 
mechanism is likely to retard the poten- 
tial speed gains of parallelization. 
In this article, I'll examine thread com- 
munication mechanisms that may be used 


Lalit is the chief technical officer for Atlas 
Software, a consulting firm based in Pitts- 
burgh, Pennsylvania. He can be contacted 
at lalitp@earthlink. net. 


ay 





dl 
Hil 
= 
: i 
. 


within parallel algorithms. To facilitate 
comparisons between these mechanisms, 
I present an abstract problem and discuss 
the different mechanisms in the context 
of this problem. The discussion is based 
on primitives provided by Windows NT. 


The Problem: 

Parallel Graph Traversal 

Performing an operation on the nodes of 

a graph is a problem that arises in a va- 

riety of contexts, including downloading 

pages rooted at a web URL up to a spec- 

ified depth, performing an operation on 

a set of files in a filesystem, and the like. 
Examples such as these involve travers- 

ing a graph, and running an algorithm on 


each node as it is being traversed. The 
trick is to do the traversal in parallel. Here, 
I will take up the traversal of a special 
case of a graph— the multiway tree. Dif- 
ferences between the two are not impor- 
tant for discussing usage of thread com- 
munication mechanisms within a parallel 
algorithm. 

The first step is to look at the sequen- 
tial traversal of a multiway tree. Listing 
One is a tree definition where each node 
contains a list of zero or more child nodes. 
The root node defines the tree. 

Listing Two implements a sequential 
algorithm for breadth-first (or level or- 
der) traversal of a tree using a queue. 
The algorithm starts by putting the root 
of the tree into the queue. Within each 
iteration, a node is taken off the queue, 
its children are enqueued, and the node 
is processed. This continues until there 
are no more children; that is, until the 
leaf nodes are reached. Use of a queue 
is what makes this traversal breadth first. 
Using a stack would have resulted in a 
depth-first traversal. 

Now I'll do the same thing in parallel. 
For this, I introduce the abstraction of a 
“concurrency engine.” Instead of directly 
putting nodes that have to be processed 
into a queue, I enqueue them into a con- 
currency engine. The concurrency engine 
is an “active” object that has multiple work- 
er threads running inside it to handle re- 
quests. Once a node is enqueued into the 
engine, a worker thread processes it in 
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(continued from page 32) 

parallel. The concurrency engine is based 
on the class template ConcurrencyEngine, 
defined in Listing Three. The template has 
two template parameters—kRequest and 
Work. These parameters let you specify a 
background task and contextual data re- 
quired by the task to do its job. Work is 
a functor whose function call operator is 
called in a background thread within the 
engine. A pointer to the instance of the 
Request template parameter enqueued into 
the engine with enque() is passed to the 
function call operator. The init method of 
the engine sets up the underlying thread 
communication channel between threads 
calling enque() on the engine and threads 
running the Work functor. 

So what exactly is a concurrency engine? 
It is an abstraction that provides a mecha- 
nism for asynchronous, concurrent execu- 
tion of tasks. The template parameters used 
to instantiate an engine let you specify a 
policy that defines what these tasks will do 
and the contextual data required by these 
tasks. You put requests into the engine by 
calling enque(). A worker thread inside the 
engine calls deque() to receive the request. 
It then calls the Work functor to run a con- 
current task based on this request. How 
the request is dispatched from a thread call- 
ing enque() to a thread calling deque() is 
up to the engine. Listing Four implements 
a parallel traversal algorithm that uses the 
concurrency engine. Compare this with List- 
ing Two. Enqueuing the root looks the 
same, but what happened to the rest of the 
stuff? Well, it’s gone into a Work functor, 
LotAlgo. The thread that initiates the traver- 
sal just needs to enqueue the root. The 
worker threads running the functor handle 
everything after that. Listing Five shows 
what they do. 

The LotAlgo functor is called by a work- 
er thread with the root node of the tree 
as a parameter. It, in turn, enqueues the 
children of this node back into the engine 
to allow a fellow worker to process these 
in parallel. The functor then proceeds to 
run a predefined unit of work on the cur- 
rent node. 

With that out of the way, I'll now turn 
to thread-communication mechanisms that 
can be used to build a channel between 
a thread calling ConcurrencyEngine:: 
enque() and one calling Concurrency- 
Engine: :dequet ). 


Thread Communication Mechanisms 

Executive objects within the NT kernel 
come in two flavors— synchronizable 
and nonsynchronizable. Synchronizable 
objects can be used to synchronize 
threads. Such objects are either in a sig- 
naled or a nonsignaled state during the 
course of their lives. A thread can do a 
wait on these objects. If the object is in 
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a signaled state, the thread’s wait is satis- 
fied. Otherwise, the thread blocks until 
some other thread (or an interrupt) comes 
along and sets the object’s state to sig- 
naled, at which point the thread is awak- 
ened by the operating system. A few of 
these NT objects—Mutexes, Events, and 
Semaphores— exist for the sole purpose 
of synchronization. Others serve a differ- 
ent purpose, but can be used for syn- 
chronization as well. Of these, one that 
will be used later is the File object. Criti- 
cal sections are lightweight objects pro- 
vided by the Win32 subsystem for fast in- 
traprocess synchronization. I refer to these 
synchronization objects as “data locks.” 
Another important class of synchro- 
nization primitives is “predicate locks.” 
Operating systems such as Solaris provide 
a built-in predicate lock known as the 
“condition variable” (see Listing Six), but 
you have to roll your own on NT. A con- 
dition variable is linked to a predicate as- 
sociated with some shared data in your 
program. Condition variables work in con- 
junction with external data locks. The 
predicate itself is not part of the condition 
variable. It is up to the threads using the 
condition variable to protect access to the 
shared data associated with the predicate 
by using the external lock. If a thread 
wants to examine the predicate to, say, 
check that the predicate is True before 
proceeding, it has to acquire the external 
lock. If it then finds that the predicate is 
False, it can do a wait on the condition 
variable. This releases the lock associated 
with the condition, allowing another thread 
to come along and change the shared 
data— after acquiring the lock. If a thread 
sets the predicate to True, it can do a sig- 
nal on the condition variable, waking up 
a thread sleeping inside the condition’s 
wait. The signaler then has to release the 
external lock to allow the waiter to run. 
Condition variables, when used in con- 
junction with plain, old-fashioned queues, 
allow the construction of an elegant 
thread-communication mechanism — the 
producer-consumer queue. 


Producer-Consumer Queues 
Producer-consumer queues provide a 
powerful mechanism for transferring in- 
formation between threads. Consumer 
threads try to get information from pro- 
ducer threads via a shared buffer. If that 
information is not immediately available, 
they go to sleep until the time that a pro- 
ducer signals availability of information. 
Similarly, producer threads try to send in- 
formation to consumer threads. If the in- 
formation cannot be buffered, they go to 
sleep until a consumer thread signals avail- 
ability of space in the buffer. 

Listing Seven implements a producer- 
consumer queue. The underlying buffer 


is protected by a mutex during updates 
in read() and write(). The same mutex is 
used to initialize two condition variables 
that signal the notEmpty and notFull con- 
ditions, which allow consumers to block 
if no information is available, and pro- 
ducers to block if the underlying buffer is 
full. Listing Eight is an implementation of 
ConcurrencyEngine::enque() and Con- 
currencyEngine.:deque() using producer- 
consumer queues. 

Though producer-consumer queues 
provide a powerful abstraction for in- 
terthread communication, they have cer- 
tain implementation disadvantages. At 
minimum, they require a predicate lock 
to signal the not-full condition, another 
predicate lock to signal the not-empty 
condition, and an external lock to pro- 
tect updates to the underlying buffer. The 
order in which consumer threads wake 
up to service requests from a producer 
is also likely to be first-in/first-out when 
operating system-supplied synchronization 
primitives are being used. This is not very 
efficient in terms of the availability of a 
thread’s working set within a system’s 
memory hierarchy, a point that is explained 
in a subsequent section. In addition, a large 
number of consumer threads are likely to 
bog down a computation-intensive algo- 
rithm. What is required is a mechanism 
for thread communication that minimizes 
locking overhead, allows for intelligent 
scheduling of worker threads, and pro- 
vides for control over the number of 
workers running concurrently on a sys- 
tem. Enter I/O completion ports. 


1/O Completion Ports 

I/O completion ports are synchroniza- 
tion mechanisms that first appeared in 
Windows NT 3.5. To use them, you cre- 
ate a file handle for overlapped or asyn- 
chronous I/O, associate the handle with 
a completion port, and have your work- 
er threads wait on the port. As soon as 
information becomes available on a han- 
dle attached to the port, a worker thread 
is woken up to handle the request. Com- 
pletion ports have significant advantages 
over other synchronization mechanisms. 
With the other NT synchronization ob- 
jects, waiting threads are released in first- 
in/first-out order. With completion ports, 
worker threads are released in last- 
in/first-out order. How does that matter? 
The explanation goes like this. The 
thread that ran last is the one most like- 
ly to have its working set in memory. On 
the other hand, the thread that has been 
waiting the longest is the one most like- 
ly to have more of its working set 
swapped out to disk. Thus, releasing 
waiting threads in last-in/first-out order 
is likely to minimize page faults and 
cache misses. 


Dr. Dobb’s Journal, April 1999 








www.parasoft.com/products/ 








SS a SS SS Se eS SE EE EE I I EE TT 


(continued from page 34) 

Completion ports also allow control over 
the number of threads associated with a 
port that can execute concurrently. So, you 
can have any number of worker threads 
waiting for requests over a port, but only 
a subset of these, as specified by the con- 
currency value of the port, will be allowed 
to execute concurrently. For example, sup- 
pose you have a four-processor system 
and you assign a concurrency value of 
four to your completion port. You start 
eight worker threads to handle requests 
for this port. Now, five requests become 
available on the port. Four of the work- 
ers can kick in and start servicing these 
requests. The concurrency value of four 
will prevent a fifth thread from waking up 
and handling the last request. But that’s 
okay, because if the fifth thread woke up, 
it would preempt one of the four running 
threads. Your overall throughput would 
probably go down because of the context 
switch overhead this introduces. Having 
the four additional worker threads on 
standby helps whenever a running thread 
does something that makes it block, like 
reading from a disk. The moment that hap- 
pens, another worker wakes up and starts 
processing the next request packet pend- 
ing in the port. 

I now need to hook up the worker 
threads within the concurrency engine to 
a completion port, and to make the 
enque() method put request packets into 
the port. Sockets offer a solution. On NT, 
socket handles are native file handles. Us- 
ing them, it is fairly easy to hook things 
up appropriately. The first step is to as- 
sociate a completion port with the engine. 
The engine constructor creates a bunch 
of worker threads to handle requests for 
this port. It also creates a manager thread. 
This thread in turn creates a TCP server 
socket, binds it to a TCP port, and starts 
waiting for connections on this socket. A 
client thread then calls init() on the en- 


gine. init() creates a client socket and con- 
nects it to the server socket. The manag- 
er thread within the engine accepts this 
connection and associates the corre- 
sponding socket with the completion port. 
Subsequently, a thread may call enque(). 
The enque() method writes to the client 
socket, at which point one of the work- 
ers sleeping on the completion port 
wakes up in the deque() method, reads 
the request packet, and services the re- 
quest. 

I now have a parallel tree traversal algo- 
rithm that uses sockets and I/O completion 
ports instead of a producer-consumer 
queue for thread communication inside 
the concurrency engine. An interesting 
feature of this algorithm is that it is pos- 
sible to run the core of the concurrency 
engine on a fast machine on the net- 
work, thus speeding up execution of the 
algorithm. 

But while this socket-based completion 
port solution is conceptually pleasing, a prac- 
tical problem remains for single-machine 
implementations. Request packets reach 
worker threads via kernel memory. In the 
write to the socket inside enque(), the re- 
quest packet is copied from user space to 
kernel space. Within deque(), it is copied 
back into user space. This is a fairly cost- 
ly operation. Here, the flexibility of com- 
pletion ports kicks in again. As it turns 
out, it is possible to bypass the usage of 
file objects when using completion ports. 
To do this, you have to post a packet to 
a completion port. This causes a worker 
thread waiting on that port to wake up, 
with access to the data that was posted. 
This is exactly what is required for the 
tree traversal algorithm. The enque() 
method posts a node to a completion port, 
and a worker thread dequeues this node 
and works on it as before; see Listing Nine. 

Notice the advantages when you com- 
pare this with the usage of a producer- 
consumer queue. First, the producer- 


consumer queue solution has to employ 
a lock to guard updates to the queue. 
There is no explicit locking (and the at- 
tendant potential for bugs) involved in the 
completion port solution. Of course, there 
is no such thing as a free lunch. Inside 
the kernel, NT would be using a lock to 
manage access to the internal queue as- 
sociated with the completion port. But the 
kernel handling of locking minimizes user- 
mode to kernel-mode switches in the case 
of completion ports. There is only one such 
mode switch within the enque()/deque() 
functions. In the producer-consumer queue 
case, there are three within each function! 
Second, the producer-consumer solution 
has to explicitly manage buffer memory, 
and enqueue-into/dequeue-from this 
buffer in a threadsafe manner. This is done 
implicitly by the NT kernel in the case of 
completion ports. 


Conclusion 

It is possible to adjust the underlying 
thread communication mechanism in a 
parallel algorithm without disturbing the 
overall structure of the algorithm. A com- 
munication mechanism based on I/O com- 
pletion ports turns out to be optimal. It is 
easy to code and is very efficient in terms 
of working set usage, context switches 
(between threads), and mode switches 
(from user to kernel). Here, I’ve looked 
at a case where the operation running 
concurrently on the nodes of a graph can 
work independently on each node. In cas- 
es where dependencies between nodes 
with respect to an operation do exist, ad- 
ditional abstractions are required to syn- 
chronize the threads taking part in the ex- 
ecution of the algorithm. 
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Listing One 


// Maltiway tree definition 
template <class T> class MTree 


public : 
// contains tree elements 
class Node 


public: 
T& operator *(); 
T* operator ->(); 


private: 
T m_val; 
vector<Node*> m_children; 


} 
// enumerates child nodes of a node 
class Enumeration 
{ 
public: 
Node& nextElement(); 
bool hasMoreElements() 


MTree(); 


“MTree(); 
// returns tree root 
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Node& root(); 


// returns parent of given node 
Node& parent (Node& child) ; 
// returns child nodes of given node 


Enumeration&é children(Node& parent) ; 


private: 
Node *m_pRoot; 


}; 


Listing Two 


// Sequential level order traversal 


// Function arguments : 


// t - tree being traversed 


// oper - the operation to be performed on each node 
template<class T, class NodeQp> 
void levelOrderTraversal(Tree<T>& t, const NodeOp& oper) 


if 


queue<Tree<T>: :Node *> q; 


q.push(&t.root()); 
while(!q.empty()) 
{ 


Tree<T>::Node* node =3D q.front(); 


q-pop(); 
oper (*node) ; 


Tree<T>::Enumeration& e =3D t.children(*node) ; 


(continued on page 38) 
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(continued from page 30) 


for(; e.hasMoreElements(); ) 


{ 
q.push(&e.nextElement ()) ; 


e.free(); 
} 
} 
// Operation to be performed on each node 
// Passed as second template parameter to the function template above 
template <class T> struct NodeOper 
{ ‘ 


LotAlgo<T, Do> algo(t, d); // LotAlgo defined in listing five 
ConcurrencyEngine<Tree<T>: :Node, LotAlgo<T, Do> > q(algo); 
algo.setEngine (&q) ; 

q.init(); 

q.enque(&t.root()); 


Listing Five 


// 


Level order traversal functor 


template <class T, class Do> 
struct LotAlgo 


a operator() (Tree<T>::Node& node) const t Troe<TSG mk 
const Do& m_work; 
Ce << *node << endl; ConcurrencyEngine<Tree<T>: :Node, LotAlgo<T, Do> >* m_q; 
y; void operator() (Tree<T>::Node* r) 

° oa hy Tree<T>: :Enumeration& e =3D m_t.children(*r) ; 
Listing T ree // enque child nodes back into the engine 
It Cncaveshes eavine for(; e.hasMoreElements(); ) 

oo <class Request, class Work> class ConcurrencyEngine m_q->enque (&e.nextElement ()) ; 
public 
ConcurrencyEngine(Worké& w) ; CPiaes eho Curtent. 208: 
int init(); e.free(); 
// put request into engine a : 
void enque(Request* r); 3 
private , 
Request* deque(); // Called by internal threads 
ioe e e e 
Y: Listing Six 
// While instantiating an engine, a functor needs to be supplied as the 2nd a eoeite theereges 
// template parameter. Its function call operator is called in the background ( 
// by the concurrency engine with a pointer to an instance of the first abtie-i 
// template parameter P Cv (HANDLE lock): 
template <class Request> struct SomeFunctor ~cy(): , 
void wait(); 
void operator() (Request *r); void dignal()% 
¥3 
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Listing Four 


// Parallel level order traversal 

// Function arguments 

// t - tree being traversed 

// 4 - the operation to be performed on each node 
template<class T, class Do> 

void level0OrderTraversalMt (Tree<T>& t, const Do& d) 
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Listing Seven 


// Producer Consumer queue 

template <class Message> class PCQueue 
{ 
private: 

HANDLE m_lock; 

Cv notEmpty; 
Cv notFull; 
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queue<Message>  m_queue; 
size_t maxSize; 
public: 

PCQueue(size_t size=3D1500) 

: maxSize(size), m_lock(CreateMutex(%, 2, @)), notEmpty(m_lock), 
notFull (m_lock) 

{ 

} 

~PCQueue () 

{ 
CloseHandle(m_lock) ; 

} 


Message read(void) 


WaitForSingleObject(m_lock, INFINITE) ; 
while (m_queue. empty () ) 
{ 


notEmpty.wait(); 


Message result =3D m_queue.front() ; 
m_queue.pop_front(); 


notFull.signal(); 
ReleaseMutex(m_lock) ; 
return result; 


void write(Message m) 

{ 
WaitForSingleObject(m_lock, INFINITE) ; 
while(m_queue.size() =3D=3D maxSize) 


notFull.wait(); 


queue. push_back(m) ; 
notEmpty.signal(); 
ReleaseMutex(m_lock) ; 


Listing Eight 


// Engine dispatch mechanism using producer consumer queue 
template <class Request, class Work> class ConcurrencyEngine 


private : 
PCQueue<Request *> m_pcQueue; 


template <class Request, class Work> 
void ConcurrencyEngine<Request, Work>::enque(Request* r) 


m_pcQueue.write(r) ; 


template <class Request, class Work> 
Request* ConcurrencyEngine<Request, Work>: :deque() 
{ 

return m_pcQueue.read (); 


} 


Listing Nine 


// Engine dispatch mechanism using completion port 
template <class Request, class Work> class ConcurrencyEngine 


private : 
HANDLE m_ioPort; // completion port for enqued requests 


template <class Request, class Work> 
void ConcurrencyEngine<Request, Work>::enque(Request* r) 
{ 
PostQueuedCompletionStatus(m_ioPort, sizeof(Request *), (unsigned 
long)r, @); 
} 
template <class Request, class Work> 
Request* ConcurrencyEngine<Request, Work>::deque() 
{ 
DWORD key, read; 
OVERLAPPED *pOv; 


GetQueuedCompletionStatus(m_ioPort, &read, &key, &pOv, ~-1); 
return (Request *)key; 


DDJ 


Dr. Dobb’s Journal, April 1999 










LES SES peacoat 





LEADIng 
Technology 
EE 
Development 
Toolkits. 


justanAd = 3 
i$ Not Going Activex 
| we ae FING. Aix ¢ 
to Cut it! jhe. «en 
OF oe © VIE Ni vo 







With over 1000 
features, more than 
any other toolkit on 
the market, visiting 
our website is the only 
way you can see just 
how powerful this 
award winning imag- 
ing toolkit is! 


FILE FORMATS 
- MORE THAN 50 - 
MOST COMPREHENSIVE 
SUPPORT AVAILABLE 
AND LOSSLESS JPEG! 


Hit the web 


and check out: 
IPEG «TOCA = s«IB PCT 


IMAGE PROCESSING TIFF MODCA WFX CMP 
SCANNING DICOM CAL MAC BMP 
COLOR CONVERSION FPX ico VDA AWD 
DISPLAY/SPECIAL EFFECTS EXF CUR GIF =WMEF 
ANNOTATIONS PSD PCK PNG EMF 
COMPRESSION PCD DCK TGA WPG 
IMAGING COMMON DIALOG EPS IMG =A: AI 
INTERNET/INTRANET . 
DATABASE For the FULL list of 
ocr File Formats and Features, 
SCREEN CAPTURE lease visit our website: 
PRINTING LEADTOOLS supports both 16 
MULTIMEDIA and 32-bit development 
MEDICAL environments, and ships with sample 
source code for Visual Basic, C/C++, 
FLASHPIX Visual C++ (MFC), C++ Builder, 
IBIG Visual J++, Visual FoxPro, Access, 


Delphi, and VB and Java script. And 
NEW support for Visual Studio 
database connectivity using OLE DB 
(JET, ODBC, Oracle and SQL 
Server) 


Includes Free Technical 
Support 





LEADTOOLS is available in several versions, 
not all features are available in all versions. 
*License required from Unisys for formats using 
LZW compression. LEAD and LEADTOOLS 
are registered trademarks of LEAD 
Technologies, Inc. All other product names are 
trademarks of their respective owners. 


Ji 





P ° R 


800-637-1840 
30-DAY MONEY BACK GUARANTEE 





a 








Simulated Recursion 





Working around 
language shortcomings 





Earl Augusta 


n the early 1960s, C.A. Hoare devel- 

oped the Quicksort sorting process. 

Quicksort divides the data into succes- 

sively smaller groups. In itself this does 
not sound very impressive, but tests show 
that Quicksort can sort large volumes of 
data tens or hundreds of times faster than 
most commonly used algorithms. How- 
ever, there is a problem. 

Quicksort uses recursion, the process by 
which a procedure or a function calls itself. 
Recursion is very powerful in attacking 
problems where the same questions are re- 
peated and the same actions are performed, 
but with each set of actions working with 
the result of a prior step. Examinations of 
data structures such as linked lists and trees 
are ideally suited for recursive program- 
ming. In languages such as C or Pascal, re- 
cursion is allowed, but languages like Cobol 
generally forbid recursion. A way around 
language limitations is to simulate recursion 
in a manner that can be coded in almost 
all programming languages. 

A recursed program examines and acts 
upon data until it arrives at a point where 


Earl is a senior consultant with Keane 
Inc., a Boston-based IT consulting firm. 
He can be contacted at earl.augusta@ 
corporate.ge.com. 
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logic suggests that there is a new set of 
data, ready to be acted upon by a new 
copy of the program being executed. At 
this point, the program calls itself. Nor- 
mally this calling passes the new data to 
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be processed by the next iteration. Even- 
tually, the logic dictates that further recur- 
sion is unnecessary, and the current active 
version of the code falls out the bottom, 
returning to execute the instruction fol- 
lowing the one that issued the call in the 
prior iteration of the recursive module. As 
each copy of the code finishes, it transmits 
information back to the calling copy. When 
control is finally passed to the original copy 
of the recursive module, falling out the 
bottom terminates the recursive process. 


Simulating Recursion 

Knowing the process by which recursion 
passes data upward and downward through 
the called modules, you can isolate and 


preserve the variables unique to each re- 
cursive step and simply loop a given piece 
of code to achieve simulated recursion. 
Since looping would start the code pro- 
cess at the same place each time, you 
must keep a place marker, indicating just 
where to begin the processing of the cur- 
rent iteration. 

The data used by each iterative step 
can be in any form and is a function of 
the programming requirements. A binary 
switch is probably the best form for a 
place marker. A combination of the re- 
quired variable data and a place marker 
form a snapshot of the conditions during 
any step of the recursion. You should con- 
struct a table where snapshots can be 
stored. A pointer to this stack of snap- 
shots allows the program to select the ap- 
propriate set of values for a particular it- 
eration of the recursive process. The 
depth of the stack must be such that it 
can contain all the steps necessary to 
achieve the objective of the program be- 
ing executed. 


Simulated Recursion 

in Quicksort Example 

The examples I present here demonstrate 
simulated recursion Quicksort in a Cobol 
program. This is one of the most hostile 
and awkward environments in which to 
do recursive programming. The idea is, 
“If we can do it here, we can do it any- 
where.” The data portion of Quicksort 
would look like Listing One. 

In Listing One, a table of 32,000 100- 
byte entries is the data to be sorted. 
SWAP-ELEMENT and PIVOT-VALUE rep- 
resent scratch work areas required to hold 
individual table entries from time to time. 
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(continued from page 40) 

The WORKING-INDEXES point to table 
entries and, as such, are full-word inte- 
gers. FURST and LAAST are pointers to the 
first and last table entries in a group cur- 
rently under consideration. The size and 
location of that group changes constant- 
ly during the progress 
of Quicksort. 

HOW-RETURN is the 
place marker enabling 
the program to begin 
processing at the cor- 
rect place when reen- 
tering the looped code. 
The place-marker con- 
cept is critical to en- 
abling fixed, looped 
code to emulate recur- 
sive processing. 

The TABLE-OF-IN- 
DEXES is a stack of 
WORKING-INDEXES. 
As the program pro- 
gresses downward (call- 
ing another iteration of 
QUICKSORT) or upward (returning to a 
prior iteration) the values for pointers and 
place markers are saved or retrieved. Every 
occurrence within the TABLE-OF-INDEXES 
is one set of WORKING-INDEXES. The 
SESSION-INDEX is the pointer to the set 
of values currently being used. LEFT and 
RIGHT-INDEX are working pointers used 
during each phase of Quicksort. 

Listing Two initializes Quicksort. It pre- 
sets the stack of working pointers to bina- 
ry zero bits. Then, one set of working 
pointers is initialized, with the FURST 
pointer set to point to the first element 
in the data to be sorted and LAAST set 
to point to the last. Setting the HOW- 
RETURN switch to 1 indicates that this is 
a new iteration of the Quicksort code and 
processing should begin at the beginning. 
The SESSION-INDEX is set to 1 (this will 
be the first session) and the data is load- 
ed onto the stack. 

Each iteration of the code in Listing 
Three first pulls data from the stack and 
then puts it into the WORKING-INDEXES. 
The value of the SESSION-INDEX deter- 
mines which data is pulled. By controlling 
the values for the SESSION-INDEX and the 
HOW-RETURN switch, you control how 
each program loop functions. 

The program logic in Listing Four ex- 
amines a specific group of table elements. 
Even though the members under con- 
sideration change continually, the ex- 
amination process is exactly the same; 
hence the application of recursive pro- 
cessing. 

The program logic establishes the first 
element to be a pivot point around which 
other comparisons will be made. Ele- 
ments are examined from left to right, 


Recursive Quicksort 
can greatly reduce 
the time needed 
to sort large 
amounts of data 


and from right to left, and element swaps 
are made where indicated. The end re- 
sult is that the pivot element has been re- 
located and now divides the group into 
two smaller groups, where all elements 
to the left of the pivot are smaller than 
the pivot, and all elements to the right are 
larger. This produces 
two new, smaller 
groups that are then 
examined in the same 
manner. 

Listing Five pre- 
pares the Quicksort 
environment to exam- 
ine first the left side, 
and then the right side 
of the two newly cre- 
ated strings of data to 
be sorted. When the 
index values have 
been set, the program 
simply calls Quicksort 
again. After Quicksort 
examines both strings 
and returns control to 
this iteration of the code, the program tests 
to see if this iteration is the first one that 
initiated the sort. If that is the case, the 
sort is complete. If not, the pointer to ses- 
sion data is reduced by one and, in effect, 
when Quicksort is called again (see List- 
ing Six), control is returned to the prior 
program iteration. 

What is the difference between a typ- 
ical user sort, where every element is 
compared against every other element, 
and a recursive Quicksort? In large vol- 
ume sorting operations, the differences 
are impressive. Recursive Quicksort could 
mean saving minutes or even hours when 
sorting large amounts of data. The re- 
cursive process is very powerful and has 
many uses in mathematical analysis. 
Quicksort is only one application of this 
type of programming logic. 


Conclusion 

While this example illustrates the use of 
a simulated recursion Quicksort in Cobol, 
this process can be used in many other 
types of languages including C and Pas- 
cal. The use of the recursive Quicksort 
can greatly reduce the time needed to 
sort large amounts of data, which can be 
useful regardless of the programming 
language. 


DDJ 


(Listings begin on page 44.) 
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(continued from page 42) 
Listing One 


@1 TABLE-TO-BE-SORTED. 
@2 TABLE-ENTRY PIC X(1@@) OCCURS 32000 TIMES. 


@1 SWAP-ELEMENT PIC X(19@). 
@1 PIVOT-VALUE PIC X(100). 


@1 WORKING-INDEXES. 


@2 FURST PIC 9(5) USAGE BINARY. 
@2 LAAST PIC 9(5) USAGE BINARY. 
@2 PIVOT PIC 9(5) USAGE BINARY. 
@2 HOW-RETURN PIC 9(5) USAGE BINARY. 


@1 TABLE-OF-INDEXES. 
@2 CURRENT-INDEXES OCCURS 100 TIMES. 
@3 FILLER PIC 9(5) USAGE BINARY. 
@3 FILLER PIC 9(5) USAGE BINARY. 
@3 FILLER PIC 9(5) USAGE BINARY. 
@3 FILLER PIC 9(5) USAGE BINARY. 
@1 SESSION-INDEX PIC 9(5) USAGE BINARY. 
* 


@1 LEFT-INDEX PIC 9(5) USAGE BINARY. 
Q1 RIGHT-INDEX PIC 9(5) USAGE BINARY. 
® e 

Listing Two 


INITIALIZE-SORT. 


MOVE LOW-VALUES TO TABLE-OF-INDEXES. 
MOVE 1 TO FURST 
HOW-RETURN 
SESSION-INDEX. 
MOVE 3200@ TO LAAST. 
MOVE WORKING-INDEXES TO 
CURRENT-INDEXES (SESSION-INDEX) . 


Listing Three 


CALL-QUICKSORT. 
MOVE CURRENT-INDEXES (SESSION-INDEX) TO 
WORKING- INDEXES . 
IF HOW-RETURN = 2 GO TO DO-PIVOT-TO-LAST. 
IF HOW-RETURN = 3 GO TO SEE-IF-WE-ARE-ALL-DONE. 


IF FURST NOT < LAAST GO TO SEE-IF-WE-ARE-ALL-DONE. 


Listing Four 


SPLIT-THE-LIST. 
MOVE TABLE-ENTRY (FURST) TO PIVOT-VALUE. 
MOVE FURST TO LEFT-INDEX. 
COMPUTE RIGHT-INDEX = LAAST + 1. 
PERFORM WITH TEST AFTER UNTIL 
RIGHT-INDEX <= LEFT-INDEX 
PERFORM WITH TEST AFTER UNTIL 
TABLE-ENTRY (LEFT-INDEX) >= PIVOT-VALUE 
ADD 1 TO LEFT-INDEX 
END-PERFORM 
PERFORM WITH TEST AFTER UNTIL 
TABLE-ENTRY (RIGHT-INDEX) <= PIVOT-VALUE 
SUBTRACT 1 FROM RIGHT-INDEX 
END-PERFORM 
IF LEFT-INDEX < RIGHT-INDEX 
PERFORM EXCHANGE-TWO-ELEMENTS 
END-IF 
END-PERFORM. 
MOVE FURST TO LEFT-INDEX. 
EXCHANGE-TWO-ELEMENTS . 
MOVE TABLE-ENTRY (LEFT-INDEX) TO ENTRY-HOLDER. 
MOVE TABLE-ENTRY (RIGHT-INDEX) TO 
TABLE-ENTRY (LEFT-INDEX) . 
MOVE ENTRY-HOLDER TO TABLE-ENTRY (RIGHT-INDEX) . 
EXCHANGE-TWO-ELEMENTS-EXIT. 
MOVE RIGHT-INDEX TO PIVOT. 


Listing Five 


DO-FIRST-TO-PIVOT. 

MOVE 2 TO HOW-RETURN. 

MOVE WORKING-INDEXES TO 
CURRENT-INDEXES (SESSION-INDEX) . 

COMPUTE LAAST = PIVOT - 1. 

MOVE 1 TO HOW-RETURN. 

ADD 1 TO SESSION-INDEX. 

MOVE WORKING-INDEXES TO 
CURRENT-INDEXES (SESSION-INDEX) . 

GO TO CALL-QUICKSORT. 


DO-PIVOT-TO-LAST. 

MOVE 3 TO HOW-RETURN. 

MOVE WORKING-INDEXES TO 
CURRENT-INDEXES (SESSION-INDEX) . 

COMPUTE FURST = PIVOT + 1. 

MOVE 1 TO HOW-RETURN. 

ADD 1 TO SESSION-INDEX. 

MOVE WORKING-INDEXES TO 
CURRENT-INDEXES (SESSION-INDEX) . 

GO TO CALL-QUICKSORT. 


Listing Six 


* 


SEE-IF-WE-ARE-ALL-DONE. 
SUBTRACT 1 FROM SESSION-INDEX. 
IF SESSION-INDEX > @ 
GO TO CALL-QUICKSORT. 
STOP RUN. 


DDJ 
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The HMAC 








Key hashing for 
message authentication 





William Stallings 


essage authentication is a procedure 
that allows communicating parties 
to verify that received messages are 
_- authentic. The two important as- 
pects are verifying that the contents of the 
message have not been altered and that the 
source is authentic. The Message Authenti- 
cation Code (MAC) is a widely used tech- 
nique for performing message authentica- 
tion. A variation on the MAC algorithm has 
emerged as an Internet standard for a wide 
variety of applications HMAC, short for 
“Keyed-Hashing for Message Authentica- 
tion.” HMAC was introduced in “Keying 
Hash Functions for Message Authentica- 
tion,” by M. Bellare, R. Canetti, and H. 
Krawczyk in Proceedings, CRYPTO 96 
(Springer-Verlag, and at http://wwwcse 
Jucsd.edu/users/mihir). HMAC is currently 
an Internet draft that has been distributed 
by the Internet Engineering Task Force as 
Request For Proposal (RFP) 2104. 


MAC 

MAC algorithms involve the use of a secret 
key to generate a small block of data, 
known as a “message authentication code,” 
that is appended to the message. This tech- 





William is a consultant, lecturer, and au- 
thor of books on data communications 
and computer networking. His most re- 
cent book is Cryptography and Network 
Security: Principles and Practice, Second 
Edition (Prentice-Hall, 1998). He can be 
contacted at http://www.shore.net/~wWs. 
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nique assumes that two communicating par- 
ties, say A and B, share K as a secret key. 
When A has a message to send to B, it cal- 
culates the message authentication code as 
a function of the message and the key. The 
message and code are transmitted to the in- 
tended recipient. The recipient performs 
the same calculation on the received mes- 
sage, using the same secret key, to gener- 
ate a new message authentication code. The 





received code is compared to the calculat- 
ed code; see Figure 1. If you assume that 
only the receiver and the sender know the 
identity of the secret key, and if the received 
code matches the calculated code, then: 


1. The receiver is assured that the mes- 
sage has not been altered. If an attacker 
alters the message but does not alter 
the code, then the receiver’s calculation 
of the code will differ from the received 
code. Because the attacker is assumed 
not to know the secret key, the attack- 
er cannot alter the code to correspond 
to the alterations in the message. 

2. The receiver is assured that the mes- 
sage is from the alleged sender. Because 


Algorithm 


no one else knows the secret key, no 
one else could prepare a message with 
the proper code. 

3. If the message includes a sequence 
number (such as is used with X.25, 
HDLC, and TCP), then the receiver can 
be assured of the proper sequence, be- 
cause an attacker cannot successfully 
alter the sequence number. 


A number of algorithms could be used 
to generate the code. The National Bu- 
reau of Standards (NBS), in its publication 
DES Mode of Operation (http://www.nist 
.gov/itl/div897/pubs/fip113.htm), recom- 
mends the use of the DES algorithm. The 
DES algorithm is used to generate an en- 
crypted version of the message, and the last 
bits of ciphertext are used as the code. A 16- 
or 32-bit code is typical. HMAC is a more ef- 
ficient, and increasingly popular, alternative. 

The process just described is similar to 
encryption. One difference is that the au- 
thentication algorithm does not need to 
be reversible, as it must for decryption. It 
turns out that because of the mathemati- 
cal properties of the authentication func- 
tion, it is less vulnerable than encryption 
to being broken. 


HMAC 

In recent years, there has been increased 
interest in developing a MAC derived from 
a cryptographic hash code, such as MD5, 
SHA-1, or RIPEMD-160. The motivations 
for this interest are: 


e Cryptographic hash functions generally 
execute faster in software than sym- 
metric block ciphers such as DES. 

e Library code for cryptographic hash 
functions is widely available. 

e There are no export restrictions for cryp- 
tographic hash functions, whereas sym- 
metric block ciphers, even when used 
for MACs, are restricted. 
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Term Definition 


Embedded hash function (MD5, SHA-1, RIPEMD-160, and so on). 


Message input to HMAC (including the padding specified in the 


embedded hash function). 


ith block of M,O <isl-1. 


Number of blocks in M. 


Number of bits in a block. 


Length of hash code produced by embedded hash function. 


Secret key; if key length is greater than b, the key is input to the 
hash function to produce an n-bit key; recommended length is 2 n. 


Padded with zeros on the left so that the result is b bits in length. 


00110110 (36 in hexadecimal) repeated b/8 times. 


01011100 (5C in hexadecimal) repeated b/8 times. 


Table 1: Definition of terms in Figure 2. 


HMAC Security 

The security of any MAC function based 
on an embedded hash function depends 
in some way on the cryptographic strength 
of the underlying hash function. The ap- 
peal of HMAC is that its designers have 
been able to prove an exact relationship 
between the strength of the embedded 
hash function and the strength of HMAC. 
The security of a MAC function is gener- 
ally expressed in terms of the probability 
of successful forgery with a given amount 
of time spent by the forger and a given 
number of message-MAC pairs created 
with the same key. In essence, it can be 
shown that, for a given level of effort (time, 
message-MAC pairs), on messages gener- 
ated by legitimate users and seen by at- 
tackers, the probability of a successful at- 
tack on HMAC is equivalent to one of the 
following attacks on the embedded hash 
function: 


1. Attackers are able to compute an out- 
put of the compression function even 
with an Initial Value (IV) that is ran- 
dom, secret, and unknown to attack- 
ers. 

2. Attackers find collisions in the hash 
function even when the IV is random 
and secret. 


In the first attack, you can view the 
compression function as equivalent to 
the hash function applied to a message 
consisting of a single b-bit block. For 
this attack, the IV of the hash function 
is replaced by a secret, random value 
of n bits. An attack on this hash func- 
tion requires either a brute-force attack 
on the key, which is a level of effort on 
the order of 2”, or a birthday attack, 
which is a special case of the second 
attack. 
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In the second attack, attackers are 
looking for two messages, Mand M'that 
produce the same hash: H(W=H(M’). 
This requires a level of effort of 2”/2 for 
a hash length of 7. On this basis, the se- 
curity of MD5 is called into question, be- 
cause a level of effort of 2% looks feasible 
with today’s technology. Does this mean 
that a 128-bit hash function such as MD5 
is unsuitable for HMAC? The answer is 
no. To attack MD5, attackers can choose 
any set of messages and work on these 
offline on a dedicated computing facili- 
ty to find a collision. Because attackers 
know the hash algorithm and the default 
IV, attackers can generate the hash code 
for each of the messages that attackers 
generate. However, when attacking 
HMAC, attackers cannot generate mes- 
sage/code pairs offline because attack- 
ers do not know K. Therefore, attackers 
must observe a sequence of messages 
generated by HMAC under the same key 
and perform the attack on these known 
messages. For a hash code length of 128 
bits, this requires 2°* observed blocks 
(273 bits) generated using the same key. 
On a 1-Gbps link, you would need to 
observe a continuous stream of messages 
with no change in the key for about 
250,000 years to succeed. Thus, if speed 
is a concern, it is fully acceptable to use 
MD5 rather than SHA-1 or RIPEMD-160 
as the embedded hash function for 
HMAC. 

Listing One, the appendix to RFC 2104, 
is sample code for the implementation of 
HMAC with MD%5. Listing Two (also from 
RFC 2104) presents test vectors for Listing 
One (trailing ‘\0’ of a character string not 
included). 


DDJ 
(Listings begin on page 50.) 
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MD5Update(&context, text, text_len); /* then text of datagram */ 


(continued from page 49) MD5Final(digest, &context) ; /* finish up ist pass */ 
/* perform outer MD5 */ 
oe MD5Init (&context) ; /* init context for 2nd pass */ 
Listing One MD5Update(&context, k_opad, 64); /* start with outer pad */ 
MD5Update(&context, digest, 16); /* then results of ist hash */ 
ia hmac_md5 */ MD5Final(digest, &context) ; /* finish up 2nd‘pass */ 
voi } 


hmac_md5(text, text_len, key, key_len, digest) 
unsigned char* text; /* pointer to data stream */ 


int text_len; /* length of data stream */ Listing Two 
unsigned char* key; /* pointer to authentication key */ 
int key_len; /* length of authentication key */ key = Px DbSbAbSbAbSbObObSbObSbObSbObSbOb 
caddr_t digest; /* caller digest to be filled in */ key_len = 16 bytes 
{ data = "Hi There" 
MD5_CTX coutext; data_len = 8 bytes 
unsigned char k_ipad[65]; /* inner padding - key XORd with ipad */ digest = 0x9294727a3638bb1c13f48ef8158bfc9d 
unsigned char k_opad[65]; /* outer padding - key XORd with opad */ 
unsigned char tk[16]; key = "Jefe" 
int i; data = "what do ya want for nothing?" 
/* if key is longer than 64 bytes reset it to key=MD5(key) */ data_len = 28 bytes 
if (key_len > 64) { digest = x750c783e6abOb503eaa86e31Ga5db738 
MD5_CTX tctx; key = OxAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 
MD5Init (&tctx) ; key_len 16 bytes 
MD5Update(&tctx, key, key_len) ; data = @xDDDDDDDDDDDDDDDDDDDD... 
MD5Final(tk, &tctx); . .DDDDDDDDDDDDDDDDDDDD... 
. .DDDDDDDDDDDDDDDDDDDD... 
key = tk; . .DDDDDDDDDDDDDDDDDDDD... 
key_len = 16; . . DDDDDDDDDDDDDDDDDDDD 
} data_len = 5@ bytes 
/* the HMAC_MD5 transform looks like: digest = @x56be34521d144c88dbb8c733f£Oe8b3f6 
* MD5(K XOR opad, MD5(K XOR ipad, text)) 
* where K is an n byte key 
* ipad is the byte 9x36 repeated 64 times 
* opad is the byte Ox5c repeated 64 times DDJ 
* and text is the data being protected 
*/ 


/* start out by storing key in pads */ 
bzero( k_ipad, sizeof k_ipad); 
bzero( k_opad, sizeof k_opad); 
beopy( key, k_ipad, key_len) ; 
beopy( key, k_opad, key_len) ; 


/* XOR key with ipad and opad values */ 
for (i=@; i<64; i++) { 

k_ipad[i] *= 0x36; 

k_opad[i] “= @x5c; 


} 

/* perform inner MD5 */ 

MD5Init (&context) ; /* init context for 1st pass */ 
MD5Update(&context, k_ipad, 64) /* start with inner pad */ 





Going im your search for 
a reliable object-oriented reverse 
engineering solution? 


Download a evaluation copy of MicroGOLD's 
WithClass at and see: 








4 
i 
44 
eg 
3 
L. 





How to get the power to accurately reverse engineer 
Visual C++, Visual Basic, Delphi, Java and IDL code. 


How integrated Microsoft Visual Basic for Applications lets you customize 
with add-ins, report generators, COM interface, ActiveX controls, script 
writing and integration with Visual Studio. 


How fast you can generate object-oriented template code from your UML models. 


The best value in an object-oriented analysis and design tool, complete 
with a 30-day money-back guarantee. 






. Featuring 
Microsoft* 


Visual Basic 


Technology 





DESKTOP REPORTING AND ANALYSIS 


Now instead of struggling with develop- 
ment tools that don’t fit, you have this: 
Seagate Crystal Reports 7, designed 
from the ground up to work seamlessly 
with Microsoft Visual Basic® Not only 
can you pull it up from within Visual 
Basic (making application development 
easier), it's so powerful and flexible it’s 
a puzzle why you'd use anything else. 


1-800-877-2334 0 


SS Seagate Software 


Wo WoW. S$ ea -9-a.t eS of tt w-apre-..c o m / pl Information, the way you want it” 


©1998 Seagate Software, Inc. Seagate, Seagate Software, Seagate Crystal Reports, and the Seagate logo are trademarks of Seagate Technology, Inc. 
or one of its subsidiaries. Visual Basic is a registered trademark of the Microsoft Corporation. Outside of the US and Canada call 1-604-681-3435. 








The PalmPilot’ 
Infrared Port 





Communication 
without cables 





A.J. Musgrove 


he PalmPilot was originally designed 

as a Personal Information Manager 
(PIM). Consequently, the software that 
came with it included a date book, ad- 
dress book, to-do list, note-taker, and the 
like. Recent versions of the PalmPilot have 
gone beyond the simple PIM model, how- 
ever. The Palm III, for instance, provides 
extensive communications support (in- 
cluding Eudora Pro, cc:Mail, POP3 Internet 
e-mail, and more) and an infrared (IR) port. 
Applications bundled with the Palm III 
support data exchange over IR. The ex- 
change of these data objects is not the 
only application for the IR port. While the 
port is not powerful enough to act as, say, 
a remote control for a television, it does 
have enough power to support Palm-to- 
Palm communications of most any kind 
of data. In this article, I explore pro- 
gramming the IR port on the Palm. In do- 
ing so, I present both an IR test applica- 
tion and a version of the venerable game 
BattleShip that can be played between two 
players via the IR port. For the most part, 
I'll focus on the IR test application. Un- 
derstanding how it works makes BattleShip 


A. is a freelance consultant and mem- 
ber of the technical staff at MCI World- 
Com. He can be contacted at musgrove@ 
ccorp.com. 
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self-evident. The development environment 
I use to implement these applications is 
Metrowerks’s CodeWarrior for Palm Com- 
puting Platform Release 5 (available from 
3Com at http://www.palm.com/). The 
source for both the BattleShip and IR test 
programs is available electronically (see 
“Resource Center,” page 5). 


The IR Port 

The IR implementation in PalmOS 3.0 
conforms to Infrared Data Association 
(IrDA) standards (http://www.irda.com/). 
The published protocol consists of a num- 
ber of layers, some required and some 
optional; see Figure 1. 

The Async Serial-IR layer, imple- 
mented purely in hardware, is a serial- 
style IR interface that supports speeds 
from 9600 to 115,200 bits/sec. The spec- 
ification itself supports synchronous 
communication at speeds of up to 4 
bits/sec., but this was not implemented 
on the Palm. 






The IR Link Access Protocol (IrLAP) lay- 
er provides reliable data transfer on a 
device-to-device basis and device discover. 
IrLAP, which corresponds to the transport 
layer of the OSI network model, supports 
a single connection between two devices. 

The IR Link Management Protocol 
(UrLMP) provides multiplexed, session- 
oriented communication on top of IrLAP. 
A component of IrLMP is the Information 
Access Service (IAS). It provides a service 
and protocol database complete with dis- 
covery mechanisms. Using IrLMP, a device 
can carry on multiple, noninterfering con- 
versations with a peer device. 

The Tiny Transport Protocol (TinyTP) 
is a lightweight transfer protocol built on 
top of IrLMP. It serves as a base for oth- 
er higher-level protocols, such as the Ob- 
ject Exchange Protocol (OBEX)— the 
only top-level protocol implemented on 
the Palm. OBEX is used by the Exchange 
Manager to transfer records between ap- 
plications, such as Addresses, Appoint- 
ments, and Applications. Palm vertically 
implemented the protocol layers neces- 
sary to implement the Exchange Manag- 
er. However, every software layer of the 
protocol, except OBEX, is exposed 
through APIs. 


The Palm’s IR Library 
The IR library on the Palm is implement- 
ed as a system library instead of as a set 
of operating-system entry points. The IR 
library is unusual; its macros and func- 
tion prototypes, for instance, are not even 
included when you use Palm.h—you 
must also include irlib.h to have these. 
The first step of using any system library 
is to locate the library entry point using 
(continued on page 56) 
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Figure 1: The IrDA protocol stack. 





Example 1: (a) The SysLibFind() prototype; (b) lrOpen() and IrClose() 
prototypes; (c) prototypes for IrBindQ) and IrUnbindQ, (@) IrCallBackQ’s 
definition; (e) IrSetDeviceQ prototype. 





Table 1: \rCallbackParms structure. 





Table 2: Events for the IrCallback() function. 
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(continued from page 52) 
SysLibFind(). Example 1(a) is the Sys- 
LibFind() prototype. 

The libName parameter is the name of 
the library you are trying to locate. The 
name is defined as irLibName in irlib-h. 
refNum is the library handle reference 
number and is a return value of this func- 
tion. It must be used in all subsequent 
calls into the library. 

Just like the rest of the Palm III’s Palm- 
OS 3.0 operating system, the IR library is 
event based. However, it does not use 
the standard event queue mechanism that 
everything else on the Palm uses. Instead, 
you must give the library the address of 
a callback function. This function is called 
to respond to every event for a commu- 
nications session. 

IrOpen() must be called before using 
any other IR library function. This func- 
tion allocates the global data areas and 
allocates system resources. When you 
are finished with the IR library, call Jr- 
Close(). These are usually called in the 
StartApplication() and StopApplication() 
functions, respectively. Example 1(b) 
shows the prototypes for IrOpen() and 
IrClose(). The options parameter of 
IrOpen() specifies the desired connec- 
tion speed. irOpenOptSpeed9600(), 
irOpenOptSpeed5 76000), and irOpenOpt- 
Speed115200() request that connections 
be opened at speeds of 9600, 57,600, or 
115,200 bits/sec, respectively. 

The connection phase of the com- 
munication introduces the notion of 
client-server. Once the connection is es- 
tablished, the two sides are in a peer- 
like relationship. The server is the pro- 
cess waiting for a connection. It can go 
about other tasks while waiting for an 
incoming connection request. The client 
is the process that initiates the connec- 
tion. Data can be transferred during the 
connection-setup portion of the con- 
versation. This is useful for doing things 
such as exchanging hardware IDs or oth- 
er setup information (which I do with 
BattleShip). 

As with most protocols, you must 
know what you want to connect to be- 
fore actually connecting. The identifying 
unit for conversations is the LSAP selec- 
tor. In both the IR test application and 
BattleShip, I exchange the LSAP selector 
during device discovery. I do this using 
the device-information feature. The lo- 
cal LSAP selector for the client is obtained 
during the protocol stack-binding phase 
of the application, which also sets up 
the callback function. The application 
must issue an /rBind() for each sepa- 
rate conversation it will carry on. Once 
an application is done with the connec- 
tion, it should call IrUnbind() to remove 
the connection from the protocol stack. 


Dr. Dobb’s Journal, April 1999 






MIccear 
‘@alste-le 


The COBE BCT Trima Automated 
Blood Component Collection System 


_— — | Quick. What's the one thing that makes an RTOS worthy of the term 
{ aes = 


“mission critical”? The engineers at COBE® BCT™ can tell 


seat icateaaten ae ees 7 Call about our Eval System! 
With our advanced rchitecture? That's right. By taking advantage of the 800 676-0566 ext. 1382 


MMU-protected OS, 
COBE Trima™ development team created a next-generation blood collection system that not only meets, 


Network" _sowoiaon, f° Device % 
eat ; Filesystem > Mihi 


advanced microkernel architecture of the QNX® RTOS, the 
prototyping, debugging, 


testing — everything — 
takes less time. but exceeds rigorous safety requirements. Better yet, QNX architecture accelerated the team’s design 
So you have more 


time to innovate. Visit ; ; . 
WWw.gnx.com/literature The result? COBE BCT delivered ahead of schedule — with a system so advanced it has an expected 


cycles dramatically. And gave them more time to innovate. 


for a free white paper. life cycle of over 10 years. Mission: accomplished. 


evecngonteqne UU URE ENGELS RE UU itil 


WWW. q nx.com The Leading Realtime OS for PCs 
cn 


© QNX Software Systems Ltd. QNX is a registered trademark of QNX Software Systems Ltd. All other trademarks and registered trademarks belong to their respective owners 


(continued from page 50) 
Example 1(c) presents the prototypes for 
IrBind() and IrUnbind(). 

The JrBind() call registers a connection 
with the protocol stack. It expects three 
parameters: the library reference number, 
a pointer to a connection structure, and 
a pointer to the callback function for the 
connection. The reference number is the 
number that came from the JrOpen() call. 
Example 1(d) is IrCallBack()’s definition. 

All events from the library are sent to 
the callback function. Table 1 shows the 
structure of the IrCallbackParms data 
structure, while Table 2 presents the dif- 
ferent event types that the IrCallback() 
routine can receive. 

The JrBind() function also fills in the 
ILsap field in the IrConnect structure. This 
is the local LSAP selector. The only field 
in IrConnect that should have set users is 
rLsap. This is the remote LSAP selector for 
the connection. All other fields in this 
structure are for system use and are 
changed by the system or macros. 

With a connection bound to the stack, 
the next logical thing to do is set your de- 
vice info — the information that’s returned 
to the other device during its discovery. 
I use a 1-byte device information string 
that is the local LSAP selector. Example 
1(e) is the prototype for the /rSetDevice() 
function. 


Client or Server? 

BattleShip’s StartApplication() function 
(see Listing One) implements all of the 
steps covered to this point. This puts you 
in a position to act as either a client or a 
server later on. 





Once you have a connection entry on 
the protocol stack, you can go forward 
as either a client or server. A server is 
easier— you just wait for the callback 
routine to be activated by an incoming 
connection request. As a client, you must 
do some connection-setup work. 

The first task is to find out to what you 
are going to connect. IrDiscoverReq() starts 
the discovery process. Example 2(a) is the 
prototype for this function. Like most of 
the IR library, this routine is asynchronous. 
The successful return of this function 
means that the discovery process has been 
started. It is not completed until an event 
is sent to the callback function with the 
results as an LEVENT_DISCOVERY_CNF 
event. Once the discovery process is com- 
pleted, the device list will be contained in 
the deviceList field of the IrCallbackParms 
structure. The device list is of type Jr- 
DeviceList, see Table 3. 

From the device list, you can obtain 
device addresses and LSAP selectors of 
prospective peer connections. You can 
then go about establishing a connection. 
Because all IrDA communication proto- 
cols are built on top of IrLAP, the first 
step in communication is starting the Ir- 
LAP connection. The 32-bit device ad- 
dress can be found in the deviceList re- 
turned from the discovery operation. 
The device address is contained in an 
IrDevicelnfo structure; see Table 4. This 
connection is started using IrConnect- 
IrLap(); see Example 2(b). This opera- 
tion cannot run after an IrDiscoverReq() 
until an LEVENT_STATUS is received 
with a status of IR_STATUS_MEDIA 
_NOT_BUSY. 


Example 2: (a) IrDiscoverReq() prototype; (b) starting a connection using 
IrConnectIrLapQ; (c) IrDisconnectIrLap()’s prototype. 






Table 3: IrDeviceList structure. 


Table 4: \rDeviceAddr structure. | 
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The return value from this function 
should be IR_STATUS_PENDING. This 
means that the connection request has 
been initiated. This does not mean that 
the connection will fail or succeed. To de- 
termine this you must wait for the call- 
back to be sent an event. If the return val- 
ue is IR_STATUS_MEDIA_BUSY, then 
there is already an active connection or 
discovery process happening on the de- 
vice. If the connection is completed suc- 
cessfully, an LEVENT_LAP_CON_CNF 
event is sent to the callback function. An 
LEVENT_LAP_DISCON_IND is sent in the 
case of failure. On the server side, the call- 
back function in the application is called 
for the event LEVENT_LAP_CON_IND. 
This is the indication to the server that the 
IrLAP connection has come up. 

Either the client or server can terminate 
the IrLAP connection with a call to /rDis- 
connectIrLap( ). Example 2(c) presents its 
prototype. If IrLAP is disconnected, the 
event LEVENT_LAP_DISCON_IND will be 
sent to all bound /rConnect structures in- 


- dicating that the connection has gone 


down. This means that all active LMP and 
TinyTP connections are also terminated. 

There are two possible return values 
for this function. IR_STATUS_ PENDING 
means that the disconnect request has 
been successfully submitted. The callback 
function will be called once the request 
has been completed. IR_'STATUS_NO_LAP 
means there was no IrLAP connection to 
terminate. 


The IR Test Application 

When running the IR test application, you 
must first select one of the Palm devices 
to be your client, and the other to be the 
server. Start the application on both de- 
vices. You should see a screen like Fig- 
ure 2. The buttons represent different ac- 
tions that can be taken against the IR stack, 
and the Send line is where you can enter 
data to send to the peer once the con- 
nection is up. 

On both devices, bind the connection 
to the protocol stack by pressing the Bind 
button. Next, on the device that you se- 
lected as the client, execute a discovery 
operation by pressing Discovery. Once 
you have an S_MEDIA_NOT_BUSY event, 
start the IrLAP connection by pressing Ir- 
LAP. Both the client and the server are giv- 
en notification that the connection came 
up. From either device, choose to dis- 
connect the IrLAP connection by pushing 
IrLAP again. Notifications are then received 
by both sides. 

With the IrLAP connection in place, you 
are set to start one of the higher-level 
protocols —IrLMP or TinyTP. Deciding 
which connection you would like to start 
is simple— you just set the parameter in 
the connection structure. For the purpose 
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of this example, I'll use IrLMP. Remember 
that the client and the server must be set 
the same, so they will both have to set the 
protocol before the connection is started. 
The protocol type is set by using the IrSet- 
ConTypeLMP() or IrSetConTypeTTP() 
functions; see Example 3(a). 

Once you have set the connection 
type, everything else about the protocol 
is automatic. No matter which connec- 
tion type you are using, the relevant 
events will begin with LEVNET_LM and 
there is no difference in the functions 
used to manipulate the connection. The 
client initiates a connection to its peer 
with a call to IrConnectReq(); see Ex- 
ample 3(b). The refNum is the library ref- 
erence number; just as in all the other 
calls. con is a pointer to a connection 
that has been bound to the protocol stack 
and has the connection type and rLsap 
selector set. packet is a pointer to the 
data packet that will be sent to the oth- 
er side along with the connection request. 
Even if you do not want to send data to 
the other side, packet must point to a 
valid IrPacket structure. credit is the 
amount of credit that will be advanced 
to the other side and is only valid for 
TinyTP connections. Since it will be AND- 
ed with 0x7f, the value must also be less 
than 127 or results will be undefined. 


"The compiler is really great and thes 


my whole life..." - Aschwin G. 







roduct. Keep up the good work!" 





"This compiler has saved me a great deal of time and | like working with it." 
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There are three possible results of a call 
to the IrConnectReq() function: 


1. IR_STATUS_PENDING means that the 
connection request has been successful- 
ly submitted and the results will be sent 
as an event to the callback function. 


The IR library on 
the Palm is 
implemented as a 
system library 





2. IR_STATUS_ FAILED is an indication of 


the rejection of the connection request. 
A connection request could be reject- 
ed because the packet size exceeds 
IR_MAX CON_PACKET for LMP con- 
nections or IR-MAX TTP _CON_PACK- 
ET for TinyTP connections. The con- 
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- Larry O. 
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nection request could also be rejected 
if the connection is already connected 
or the /rConnect structure is not bound 
to the stack. 

3. IR_STATUS_NO_IRLAP will be returned if 
there is no established IrLAP connection. 


Whether or not the connection suc- 
ceeds, the connection information pack- 
et is sent to the callback function with the 
event LEVENT_PACKET_ HANDLED. This 
is important because the memory occu- 
pied by packet is owned by the stack un- 
til this event is sent. If you use this pack- 
et before you get this event the results will 
be undefined and probably unwanted. 

Once the connection request has been 
processed, the results will be sent to the 
callback function as an event. An event of 
LEVENT_LM_DISCON_IND indicates that 
the connection failed. LEVENT_LM_CON 
_CNF indicates that the connection suc- 
ceeded and the IrCallbackParms structure 
will contain data returned from the other 
side as part of the connection response. 

An LEVENT_LM_CON_IND event is 
generated on the device that receives the 
connection request. The connection in- 
formation from the peer device is includ- 
ed in the IrCallbackParms structure. To 
accept the connection, the application 
should issue an IrConnectRsp(); see 
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Example 3(c). This looks similar to the Jr- 
ConnectReg parameters, and each param- 
eter has the same meaning. The packet 
parameter is the data that will be returned 
to the peer to accept the connection. You 
cannot use the packet until you receive 
the LEVENT_PACKET_HANDLED. 

Use the IR test application to experi- 
ment with IrLMP. First, you must bind a 
connection, then do device discovery from 
the client, and finally start an IrLAP con- 
nection. Only then can you start an IrLMP 
connection. 

Once you have the connection estab- 
lished, you will probably want to have a 
conversation. This is accomplished by us- 


IR. Test App 





Figure 2: Running the IR test 
application. 





ing the IrDataReq(); see Example 3(d). 
This function transfers a data packet to 
the peer over the existing connection. The 
maximum size of the data packet can be 
found using IrMaxTxSize(). You must 
check InMaxTxSize() for each connection, 
as each one could have a different maxi- 
mum packet size. 

The refNum parameter is the library ref- 
erence number and the con parameter is 
the pointer to the /rConnect structure rep- 
resenting the connection. The packet pa- 
rameter is a pointer to an /rPacket struc- 
ture which contains the data to be sent to 
the peer. The possible return values of Jr- 
DataReq are IR_STATUS_PENDING or 
IR_STATUS_FAILED. 

IR_STATUS_ PENDING indicates that the 
request has been accepted by the stack 
for delivery. Once the packet has been 
delivered, the callback function receives 
the event LEVENT_PACKET_HANDLED 
and the packet is in the packet structure. 
IR_STATUS_FAILED could be returned if 
the JrConnect structure is not bound to 
the stack, the packet exceeds the maxi- 
mum size, or the JrConnect structure does 
not represent an active connection. 

When data is received, an LEVENT 
_DATA_IND event is generated. The mBu/ff 
and men parameters contain the received data. 
There is no response that should be sent to this 
event, as the stack has already handled the re- 


ceipt acknowledgment. 


Example 3: (a) Using the IrSetConTypeLMP or IrSetConTypeTTP functions; (b) 
initiating a connection peer with a call to IrConnectReq; (c) accepting the 
connection via IrConnectRsp; (d) imitating a conversation using IrDataReq; (e) 
IrLocalBusy prototype; () prototypes of functions available for protocol stack. 





Example 4: The IrTestReq function. 
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The IR test application can be used 
to demonstrate the data transfer. To use 
it, first establish an IrLMP connection 
to a peer. Then, enter the string you 
would like to transmit into the Send 
field and press the Data button. The 
data will appear in the Rec field on the 
other device. 

During processing, it may be neces- 
sary for an application to stop receiving 
data. If you are using IrLMP and only one 
connection exists, you can set the busy 
state on the protocol. If more than one 
connection exists, never set the busy state. 
Likewise, never set the busy state for 
TinyTP. Busy is represented by the device’s 
transmission of Receive Not Ready (RNR) 
frames. The JrLocalBusy() function sets 
and unsets the busy state. Its prototype is 
in Example 3(e). The flag variable should 
be True to set busy, and False to unset it. 

There are five functions provided that 
view different aspects of the protocol 
stack. Example 3(f) presents their pro- 
totypes. All of these functions expect 
the library reference number, and /r- 
MaxkxSize() expects the connection to 
which you are referring. The /rlsLap- 
Connected() function returns True if 
there is a currently active IrLAP con- 
nection. JrisMediaBusy() returns the 
busy status of the medium. If the medi- 
um is busy, any connection or discov- 
ery requests would fail. JrlsNoProgess() 
returns True if there is no progress be- 
ing made on the IrLAP connection. If 
the peer device is removed from the 
transmission range of the IR port, then 
progress could cease on data transfers. 
IrlskemoteBusy() returns True if the peer 
device is currently transmitting RNR 
frames indicating that it does not wish 
to receive data. IrMaxkxSize() returns 
the maximum data packet size that you 
can expect to receive. The value is valid 
only for the connection used in the func- 
tion call. 


Testing the IR Connection 

The final part of the Palm’s IR API involves 
testing. You can initiate a test packet to 
the peer device to see the link status. This 
is done using the /rTestReq() function; see 
Example 4. The TEST packet can only be 
sent when the stack is in the Normal Dis- 
connect Mode (NDM) state. Generally, this 
means that IrLAP cannot be connected 
and discovery operations cannot be in 
progress. 

The refNum parameter is the library 
reference number. The devAddr param- 
eter is the 32-bit address of the device 
to which the test frame should be trans- 
mitted. con is a pointer to an IrConnect 
structure whose callback function will be 
the recipient of the test status. packet is 
a pointer to an /rPacket that will be sent as 
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the test data. There are three possible return 
values: 


1. IR'STATUS_PENDING means that the 
test packet has been accepted by the 
protocol stack. The callback function is 
sent an LEVENT_TEST_CNF once the 
test has been completed. The status field 
contains IR_STATUS_SUCCESS to indi- 
cate success or IR_STATUS_FAILED to 
indicate a test failure. 

2. IR_'STATUS_MEDIA_BUSY means that 
the media is not in the NDM state and 
cannot start a test. 

3. IR_STATUS_FAILED could be returned 
if the IrConnect structure was not 
bound to the stack or the packet size 
exceeds the maximum size. 


On the server side, you will receive an 
LEVENT_TEST_IND event. rxBuff and 
rxLen in IrCallBackParms contain the 
test packet. The /rPacket structure con- 
tains the response packet. By default, it 
will contain the same packet that was 
sent as the test. However, you can change 
the data and that is what will be returned. 
Once the callback function returns, the 
test response will be sent. 


Conclusion 

This covers the IR API on the Palm III. 
If you carefully study the IR test appli- 
cation, you should have little trouble un- 
derstanding the BattleShip application. 
I'd like to add a special thanks to Todd 
Warren at MCI WorldCom for his gener- 


1 SS. 


ous loan of the additional hardware used 
in development. 
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e e 
Listing One 
static void StartApplication(void ) 


// check for ir 
if (SysLibFind(irLibName,&irref) != @) 
{ 


irAvail = false; 


else 
{ 
if (IrOpen(irref,irOpenOptSpeed115200) != 0) 
irAvail = false; 
else 
{ 
irAvail = true; 
IrBind(irref,&connect,BSIrCallBack) ; 


} 
} 
if (!irAvail) 


FrmAlert (NoIRAlert) ; 


SysGetROMToken(%,sysROMTokenSnum, &serialNo, &serialNoLen) ; 
SysRandom(TimGetSeconds()); 


game.recno = 65535; 


game.started = false; 


CurrentView = StartupFormForm; 
CurrentMenu = MenuInit (StartupMenuBar) ; 
FrmGotoForm(CurrentView) ; 


OpenDatabase() ; 


IrSetDeviceInfo(irref, (BytePtr) &connect.1Lsap,1); 


IrSetConTypeLMP (&connect) ; 
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Coping with 
unpredictability 





Rene Schaad 


“situated agent” is a controller lo- 

cated in a dynamic, complex, and 

relatively unstructured environment. 

Situated agents can be either hard- 
ware or software (or both). Software is 
almost always involved; hardware is need- 
ed to complete the agent if its environ- 
ment is physical. Hardware then plays the 
role of the interface, although it can also 
be a part of the computation itself. Typ- 
ical hardware-based situated agents in- 
clude mobile robots, autonomous un- 
derwater vehicles, intelligent rooms, and 
the like. 

Pure software situated agents, on the 
other hand, are located in virtual worlds. 
A software-based situated agent’s envi- 
ronment is logical instead of physical. The 
interface consists of communication in- 
terfaces to this logical environment — one 
that is dynamic, unpredictable, large, and 
complex. Situated software agents include 
load balancing in telecom networks, and 
virtual characters in storytelling systems 
and games. 


kene completed his Ph.D. in computer sci- 
ence at the Artificial Intelligence Labora- 
tory of the University of Ztirich, Switzer- 
land. He can be reached at schaad@ 
ifi.unizh.ch. 
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In general, agents attempt to achieve a 
multitude of goals by interacting with their 
environment via their sensors and actua- 
tors. The goals are often in competition 
with each other, and it is up to each agent 











to autonomously find a way to deal with 
the conflicts that ensue. This is a difficult 
task. The complex and dynamic nature of 
the environment has the effect that the 


agent: 


¢ Does not have complete control over 
the environment. 

e Does not have the capacity to devise 
complete models of its environment. 

¢ Does not possess complete information 
about the environment. 

e Cannot completely trust the information 
it does have, because it is usually un- 
certain, imprecise, noisy, or outdated 


due to the nature of its perceptual pro- 


Cesses. 





Decision Trees for 
Situated Agent Control 


An agent must act coherently if it is to 
achieve its goals; that is, its actions must 
consistently lead toward its goal Cif the 
agent is a mobile robot, for example, 
reaching a particular location in the envi- 
ronment). On the other hand, the agent 
must remain responsive and react appro- 
priately to unexpected events that occur 
in the environment (for instance, if unex- 
pected obstacles occur in the robot’s path). 
However, it is usually impossible to de- 
vise rules for conflicting goals such as 
these, without assuming perfect knowl- 
edge and planning. To deal with this prob- 
lem, designers of situated systems must 
be given tools to represent reactive rules 
as well as coherent actions in their plans 
(sequences). 

Because an agent does not usually have 
all the information it needs, it must act to 
gain information (as opposed to acting to 
reach its primary goals). 

Finally, using instructions sensibly in a 
given situation involves checking whether 
or not the application of these instruc- 
tions makes sense (guarded interpreta- 
tion) and in what way they bear upon 
the current situation (context- dependent 
interpretation). 

Due to a shift of emphasis in this field 
from building disembodied minds (chess 
players, for instance) to creating more em- 
bodied forms of intelligence (“elephants,” 
as described by Rodney A. Brooks in “Ele- 
phants Don’t Play Chess,” Robotics and 
Autonomous Systems 6, 1990), these types 
of control problems have recently gained 
interest in the field of Artificial Intelligence 
(AI). The effect of this shift is that some 
areas of AI have begun to overlap aspects 
of embedded-systems programming. 
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(continued from page 62) 

Many areas apply and test techniques 
from situated-agent design. Probably the 
most recognized area is that of mobile 
robotics. NASA’s Sojourner Mars rover has 
arguably been the most prominent (semi- 
autonomous) situated agent in recent 
years. But there are also other more 
down-to-earth applications — network 
load balancing, intelligent rooms, intelli- 
gent artificial creatures for virtual reality, 
and more. 

In this article, I will present an approach 
to programming reactive situated agents 
that is based on parallel functional deci- 
sion trees. In the process, I will present 
“InSitu,” a C++ class library and run-time 
system for situated agents that addresses 
the aforementioned problems and re- 


quirements. An alpha version of InSitu is 
available electronically, see “Resource Cen- 
ter,” page 5. This early version of InSitu 
is written in Watcom C++ for the QNX 4 
real-time operating system. 

In my approach to the problem of pro- 
gramming reactive situated agents, the first 
design decision I made was to implement 
a reactive run-time system in C++ and then 
create C++ classes to add functionality for 
coherence, active perception, and situat- 
ed instruction use. The result was InSitu. 
Most environments for embedded systems 
are designed the other way around— they 
use a sequential programming language 
(such as C++) and add multithreading, in- 
terrupt, and event-handling mechanisms 
to achieve reactive behavior in that se- 
quential (coherent) framework. 
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InSitu was developed and tested on a 
mobile robot called Rufus T. Firefly (see 
Figure 1 and http://www. ifi.unizh.ch/ailab/ 
projects/rufus/) that was tasked to patrol 
a floor in an office building and report hu- 
man intruders back to a base station via a 
radio link. I have found InSitu to be a mod- 
ular, efficient, and robust tool for the de- 
velopment of control laws for this type of 
situated agent. The decision tree-based ar- 
chitecture is particularly well suited for the 
incremental and fast development of 
situation- dependent control rules. 


Stepped Execution 
A run-time system for reactive situated 
agents (or any embedded system) may be 
implemented according to one of two 
types of execution models. The system ei- 
ther uses interrupts to produce events, 
which then steer event loops, finite state 
machines, and other such constructs to 
control the agent; or the system may reg- 
ularly poll its sensors, categorize the sensed 
situation, and produce outputs accord- 
ingly. Mainly for efficiency reasons, most 
systems use an event-driven approach. 
However, in the AI arena, situated agents 
are most often based on polling. The AI 
community has adopted the rather gen- 
eral term “reactive systems” for these types 
of polling-based situated agents so that 
they can be distinguished from tradition- 
al AI planning and control systems that 
emphasize coherence over reactivity and 
are often rather unresponsive. 

Control following the polling-based ex- 
ecution model, which I will call the 






Figure 1: Rufus T. Firefly. 


“stepped execution model,” proceeds in 
a loop with three steps: 


1. Based on sensor readings, a simple in- 
ternal model of the environment is up- 
dated. 

2. The state of this model is categorized 
into one of a number of situations. A 
situation is a set of states. 

3. The situation is mapped to some actu- 
ator output, and the loop recommences. 


It is important that the types of model- 
ings and mappings that are employed in 
this loop be limited to those that can be 
computed incrementally during the time 
of one loop, and that outputs are pro- 
duced at each incremental step. Long or 
unpredictable computation times must be 
avoided. Algorithms based on lookup ta- 
bles, neural networks, fuzzy logic, or de- 
cision trees are ideal candidates for this 
type of computation. 

At first glance, polling seems to be a 
bad choice. Event-driven control is usu- 
ally more efficient because computations 
are carried out only when important events 
happen. In addition, by employing mul- 
titasking, other tasks can be tended to if 
no events are being processed, thereby 
taking full advantage of each processor 
cycle. Event-driven control is also more 
convenient to program because it is usu- 
ally not necessary to divide the computa- 
tion into short incremental steps. When 
an event occurs, all the computation as- 
sociated with that step is carried out at 
once; long running processes are auto- 
matically sliced by the multitasking mech- 
anism. 

However, polling has a critical advan- 
tage. Interesting events in the environ- 
ment cannot always be assumed to be 
producing events because, as I’ve out- 
lined earlier, a situated agent does not 
have continuous access to all the impor- 
tant information. Hence, it is not sufficient 
for the agent to passively react to events 
and create outputs only in response to 
these events. It must pursue the acquisi- 
tion of information. Furthermore, it must 
produce actuator commands at regular in- 
tervals even if nothing interesting has hap- 
pened. Event-driven execution is okay for 
passive reactive systems that have con- 
stant access to all relevant information, 
but polling is necessary for systems that 
must initiate action or deal with incom- 
plete information. 

An execution system for this model can 
seem almost trivial. It consists of a loop 
with a call to the input (sensors to input 
buffers), mapping (input to output buffers), 
and output functions (output buffers to ac- 
tuators), with a delay statement. All the in- 
teresting computations take place in the 
mapping function. 
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Steppables 

The most important abstraction in InSi- 
tu is “Steppables”’— objects that encap- 
sulate mapping algorithms executed in 
a step-wise fashion, but produce coher- 
ent sequential behavior over the course 
of many steps. Thus, Steppables are de- 
signed to address the coherence-reac- 
tivity problem. According to Erich Gam- 
ma et al. in Design Patterns: Elements of 
Reusable Object-Oriented Software (Ad- 
dison-Wesley, 1995), encapsulating al- 
gorithms in objects follows a design pat- 
tern called “Strategy” and serves to make 
different algorithms compatible and in- 
terchangeable. Steppables are imple- 
mented as classes derived from the ab- 
stract class Steppable, which defines their 
basic interface. This interface consists of 


the Step, Reset, Abort, and IsDone mem- 
ber functions: 


Data *Step(void) 
void Abort (void) 
void Reset (void) ; 

Fuzz *IsDone (void) ; 


The Step member function advances the 
algorithm by one step, produces side-ef- 
fects to other Steppables and returns a re- 
sult to the caller. Reset and Abort set the al- 
gorithm’s progress to its beginning and end, 
respectively. sDone returns a fuzzy Boolean 
value, indicating the algorithm’s status (ei- 
ther it has not yet started, it is in progress, 
or it has terminated). A stepped algorithm 
may take an arbitrary number of calls to 
the Step member function for its comple- 
tion. Thus, Steppables offer support for 
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combining reactive and coherent aspects 
of execution by uniting the stepped ex- 
ecution model with the abstraction of se- 
quential algorithms. 

As an example, consider TimedIf, a 
Steppable from the InSitu class library: 


Steppable *a = new TimedIf 
(Steppable *cond, 
Steppable *then, 
Steppable *else, 
int stick_delay) ; 


TimedIf is a class derived from Step- 
pable. Its purpose is to encapsulate the 
following behavior: On each time step 
(that is, on each call of the run-time sys- 
tem to TimedIf’s Step member function), 
TimedIf calls the Step member function 
of the Steppable pointed to by the con- 








structor’s argument cond. If that call re- 
turns a Data object of type Fuzz (a kind 
of fuzzy Boolean value), and if that re- 
turn value evaluates to a true statement, 
the Step function of the Steppable point- 
ed to by then is executed. Otherwise, the 
Step function of else is executed. In addi- 
tion, after a successful execution of cond’s 
Step function, a countdown timer is initi- 
ated to stick_delay milliseconds. While 
this timer is running, then’s (rather than 
else’s) Step function continues to be ex- 
ecuted on each consecutive time step. 
Hence, TimedIf implements a kind of de- 
bounced if statement. 

TimedIf is an example of a Steppable 
intended to produce actions by calling an- 
other Steppable’s Step functions. These 
types of Steppables are called “Action- 
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Steppables” and are derived from the ab- 
stract class Action, which is itself derived 
from Steppable. Aside from Action- 
Steppables, there are also Data-Steppables, 
intended to passively store data. To this 
end, each Data-Steppable has a Step mem- 
ber function that does nothing more than 
return a pointer to itself. Abort and Reset 
are degenerated functions that do noth- 
ing, and JsDone always returns a point- 
er to a Fuzz object, which evaluates to 
True. In addition to the standard inter- 
face, Data-Steppables add functions for 
duplication, printing, accessing instance 
variables, and managing run-time type 
information. Data-Steppables may be 
passed around between Action-Steppables 
as results or parameters. They may be 
used in place of Action-Steppables (be- 
cause their interfaces are compatible), and 
they may be used from the InSitu Script 
(ISS) online instruction compiler. 

Programming in the stepped-execution 
model can be tedious. Steppables are in- 
tended to make this task easier by pro- 
viding a uniform framework, a run-time 
system, a uniform set of interface func- 
tions, and a library of premade compo- 
nents for developing, assembling, and 
running such stepped programs. Fur- 
thermore, Steppables can be dynamical- 
ly allocated, deleted, inserted, and re- 
moved from a running program. InSitu 
uses this feature to allow the ISS compil- 
er to add and remove parts of a program 
at run time. Finally, using objects to en- 
capsulate algorithms allows multiple 
copies of the algorithms with different lo- 
cal states (for example, multiple copies 
of TimedIf with different values of the 
countdown timer). These are the advan- 
tages of implementing stepped algorithms 
as objects instead of plain functions. 


Parallel Functional Decision Trees 

The constructors of Steppables accept 
pointers to other Steppables as arguments. 
These pointers are kept in instance vari- 
ables and are used by the objects’ mem- 
ber functions (Step, Abort, Reset, and Is- 
Done) throughout their lifetimes to access 
the referenced objects’ member functions. 
In this way, nested structures of Steppables 
referencing other Steppables can be con- 
structed at run time by allocating and 
cross-linking objects. 

In essence, assemblages of linked Step- 
pables implement a kind of decision tree. 
Indeed, each Steppable itself can be 
thought of as implementing a part of a de- 
cision tree, called a “decision subtree.” In 
this analogy, the root of a subtree imple- 
mented in a Steppable is represented by 
the call to its Step member function. The 
leaves of the subtree, which are at the same 
time the roots of connecting subtrees, cor- 
respond to the exits of the Steppable. An 
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exit of a Steppable is a pointer to another 
Steppable that is passed to its constructor. 
In other words, the exits of a Steppable are 
those arguments in its constructor that are 
pointers to other Steppables. 

But Steppables are not pure decision 
trees. They are enhanced in several ways. 
Their first enhancement concerns their de- 
cision functions — the functions in the 
nodes of the decision tree that select one 
of the branches for traversal. In simple bi- 
nary decision trees, this function is a 
Boolean expression, such as door == 
open. Depending on the outcome of this 
expression (True or False), one of the 
branches emerging from that node is tra- 
versed, and execution continues at the 
node at the end of that branch. 

However, the problem with this ap- 
proach in situated agents is that it is gen- 
erally not known whether the decision 
function (door == open) evaluates to True 
or False, as the agent will simply not have 
sufficient information to answer this ques- 
tion. Thus, it is necessary to allow the agent 
to answer the question by interacting with 
the environment (active perception). Since 
Steppables control interaction with the en- 
vironment, the decision function itself must 
be implemented as a decision subtree con- 
sisting of Steppables. From this, several re- 
quirements for the so-called parallel func- 
tional decision trees follow: 


e Decision functions as decision trees. De- 
cision functions must be implemented 
as decision trees as well. Consequent- 
ly, there will be no conceptual distinc- 
tion between branches that serve to 
compute a decision function, and 
branches that primarily serve to perform 
actions. Only their roles differ. In par- 
ticular, both types of decision subtrees 
may read sensors and perform actions. 

e Functional extension. Decision trees 
must be able to return data to the callers 
of their Step functions to serve as deci- 
sion functions. We will later see that the 
capability to return data is also used for 
other types of active perception. 

e Parallel extension. Decision trees must be 
able to traverse several paths on each time 
tick; for example, the path of a decision 
function, and the path of the branch se- 
lected by the result of the decision func- 
tion must be traversed in the same step. 
Even though they are executed sequen- 
tially within the time tick, conceptually 
they are executed in parallel. 


All of these requirements are met by Step- 
pables, which, in effect, implement sub- 
trees of parallel functional decision trees. 

Example 1, a simple reactive program 
implemented as a collection of decision 
trees, is a fragment of a C++ program that 
allocates five objects which are all derived 
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from the abstract class Steppable: Stickyif, 
BumpedQ, Prog, Moveby, and Turnby. It 
is a simple solution to the problem of get- 
ting a mobile robot around an obstacle. 
The program basically says: If you get 
bumped against an obstacle, perform the 
following steps to completion: Move the 
robot backwards by 0.1 meter, and then 
turn the robot by 90 degrees to the left. 
After this, or if the robot is not bumped 
against an obstacle, continue with the de- 
cision tree pointed to by do_the_other_stuff 
as a control program. 7he_robot is a point- 
er to a plant proxy; that is, an object rep- 
resenting the controlled robot that han- 
dles all input and output. The plant proxy 
is also derived from Steppable. 

After the assignment, unbump is a 
pointer to a parallel functional decision 
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tree consisting of a number of objects de- 
rived from Steppable, which can be 
stepped to execute the desired actions. 
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This is usually done by passing unbump to 
the run-time system, which then inserts it 
into a larger decision tree in a suitable spot. 

Remember that even though this frag- 
ment looks sequential and each Steppable 
controls a sequential process, it is exe- 
cuted in a purely reactive, step-wise man- 
ner. The Steppables serve to reintroduce 
sequentiality and coherence into a basi- 
cally reactive scheme. To this end, they 
store the local sequential state that sur- 
vives calls to their Step functions; that is, 
the Prog class (a member of the InSitu 
standard class library) is designed to ex- 
ecute its exits in sequence. This means 
that when Prog is stepped, it propagates 
the call to one of its exits. Each exit keeps 
being stepped until its IsDone methods 
returns True, indicating its completion, 
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at which time the program counter in 
Prog is incremented to the next exit. Af- 
ter stepping all exits (in the example a 
Moveby and a Turnby object), the default 
exit (in the aforementioned example set 
to NULL) is stepped until a call to Prog’s 
Reset function restarts the sequential pro- 
cess. Similar classes exist in InSitu’s class 
library for loops, sequences with condi- 
tional increments, finite state machines, 
and many other temporally coherent 
types of behaviors. 


Classification of Situations 

Decision trees classify data. When deci- 
sion trees are used to control an agent, 
their purpose is to classify the state of the 
world as they encounter it and assign pro- 
grammed actions to this class of world 
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states. I’ll call classes of world states “situ- 
ations.” With decision trees, this classifica- 
tion is recursive; that is, each branch in the 
decision tree adds accuracy to the classifi- 
cation, and the number of states in the re- 
sulting situation decreases with each 
branch. Each node in the decision tree thus 
corresponds to a particular situation in the 
world. By attaching a decision subtree to 
a particular node in the tree, you tell the 
system that this subtree implements a good 
strategy for this particular situation. Hence, 
when an agent stops being used, and an- 
other more appropriate one is used. 

Another way of looking at Steppables 
is that they allow you to provide a pre- 
fabricated system decision tree, called a 
“default tree,” where a situated agent de- 
fines a default classification, and thus a 
default behavior. You may then incre- 
mentally add decision trees to handle cer- 
tain situations differently, or add new dis- 
tinctions to handle subsituations in 
different ways. This possibility of incre- 
mentally adding modular handlers for sit- 
uations is a powerful tool for building re- 
liable situated agents. 


InSitu Script 

With InSitu, it is even possible to add 
such handlers (also known as “decision 
trees” or “clusters of Steppables”) during 
the run time of the system. The InSitu 
Script (ISS) compiler is an object within 
the InSitu run-time system that accepts 
commands from an online user during 
run time via a number of input channels 
(for instance, a wireless radio link on a 
mobile robot). ISS’s syntax is based on 
LISP. But in addition to pure LISP func- 
tions, InSitu also provides a mechanism 
called “factory functions.” Factory func- 
tions allocate new Steppables inside the 
InSitu system. Other system functions al- 
low you to add or remove these allocat- 
ed Steppables to and from the running 
system; see Example 2. 

The first nested call to the LISP func- 
tions creates a nested cluster of Step- 
pables. Each of these LISP functions (ex- 
cept for setq) is a factory function that 
creates an object of the corresponding 


(setq y 
(StickyIf 
(BumpedQ) 
(Prog 
(Noop) 
(Moveby -@.1) 
(Turnby 1.7) 


) 

(Noop) )) 
(Do y "some location") 
(Stop y) 


Example 2: Adding and removing 
allocated Steppables to and from the 
running system. 


Dr. Dobb’s Journal, April 1999 


microsoft.com/jobs 


Terdvosk . ~—C Microsoft 
Product Design Lead = | . 


Where do you want to go today?® 


©1999 Microsoft Corporation. All rights reserved. Microsoft and Where do you want 
to go today? are registered trademarks of Microsoft Corporation. Microsoft is an equal 
opportunity employer and supports workplace diversity. 








CT EP EEE EES EEN TERENAS SEES ETRE SETS ESET EEL CEE EE ENE ESE E ELE ETE EERE EEE ELLE EAE EDEL LEE ELLE LL LE LE LLL EEL LL 


(continued from page 08) 

type and returns a reference to that ob- 
ject. The resulting reference is stored in 
the LISP variable y. The function calls 
(Do y "some location") and (Stop y) in- 
sert and remove the new subtree into and 
from the default decision tree at a loca- 
tion designated with a so-called “Dock- 
Steppable,” which bears the name “some 
location,” respectively. Dock-Steppables 
are specifically designed to serve as 
named location for subtree insertion. All 
Docks in a system are managed by a 
Dock manager object. 

In this way, a running system may be 
changed and its dynamic behavior debugged 
by observing its actual interaction with the 
real world. ISS can also be regarded as a 
high-level command language for real-time 
man-machine communication. 


Active Perception 

Decision subtrees are able to return data 
to their callers (functional extensions). 
This facility was introduced to allow de- 
cision functions to acquire information 
through action (that is, through active 
perception) and use this information to 
select branches in decision trees. This 
type of active perception is often said to 
be about achieving knowledge goals. It 
is the goal of the agent to gather knowl- 


edge. But there is also another type of 
active perception that is supported by 
this mechanism— symbol registration. 
InSitu uses objects called “Markers” (a 
type of Data- Steppable) as references 
to objects in the real world. For instance, 
an object storing the coordinates of a 
dog in the camera image of a mobile 
robot might be such a Marker. Markers 
must be registered and tracked — their 
initial and changing values must be de- 
termined. Both registration and tracking 
may involve actions; for instance, it might 
be necessary to pan the robot’s camera, 
turn its base, or move forward to keep 
the dog in its field of view. InSitu has 
Steppables to support both registration 
and tracking. 

The programs to implement such reg- 
istering and tracking behavior are, of 
course, also implemented as Steppables, 
and the results (the markers) are com- 
municated to their callers as return values 
of their Step member functions. 


Conclusion 

A number of ideas for extending or mod- 
ifying this scheme, however, have come 
up as well. For example, decision trees 
are an ideal basis for a development en- 
vironment based on visual programming. 
But to support visual programming, the 


representation of the decision tree struc- 
ture should be separated from the Step- 
pables responsible for the executable 
code. Currently, the representation of the 
decision tree structure, the traversal algo- 
rithm for the decision tree, and the rep- 
resentation of the executable code are all 
the responsibility of the Steppables. To 
have better access to these data structures, 
visual programming would require the 
separation of these responsibilities. Fur- 
thermore, ideas are in the pipeline for lan- 
guages based on the aforementioned 
ideas, for the integration of planning al- 
gorithms (automatic program generation), 
and for natural language interfaces based 
on the types of active perception report- 
ed here. 

Parallel functional decision trees as im- 
plemented by InSitu are a simple yet 
powerful way to program reactive situ- 
ated agents. It tackles problems of the 
coherence-reactivity trade-off and of ac- 
tive perception in a pragmatic way, and 
offers a modular and efficient program- 
ming paradigm. 


DDJ 








Essential Books on File Formats CD-ROM 


} °° no further! The Essential Books on File Formats CD-ROM contains the 


complete text from six books, which will provide you with the most 
file formats in use today. Selected by the editors of Dr. Dobb’s Journal, this 
CD-ROM contains invaluable information on file formats used for 
sraphics, multimedia, sound, databases, spreadsheets, Windows, the 


Internet, and much more! 


full-text search engine 
and hyperlink 
capabilities allow 
you to search quickly 
and easily across all 
the books to link 
directly to the 
information 

you need. 


Text from 6 books 


70 


No matter what your programming focus, the Essential Books on File 
Formats CD-ROM discloses the secrets and insider knowledge you need to 
make your programming instantly compatible with all the major 
applications out there right from the start. Plus, the CD-ROM's powerful 






Dr. Dobb’s 


LIBRARY 


Urder Today! 
1 (} (} = g g ? =| HAY E-mail: orders@ mfi.com 


U.S. & Canada 





(te ee ee a ee ee 








Uy Dabs Fesential Beoke Gn 


FILE FORMATS 
















Sn in oe ia ale 


PRICE! $69.95 


WWW.DDJ.COM/CDROM/ 




















All Other Countries: 
785-841-1631 







Fax Orders: 785-841-2624 


Dr. Dobb’s Journal, April 1999 








— 
ce 








sos INTERNET PROGRAMMING 


Examining PerLDAP 





Simplifying 
LDAP access 





Troy Neeriemer 


DAP (short for “Lightweight Directo- 

ry Access Protocol”) promises to be 

a central repository of information 

about users and corporate resources. 
However, if it is difficult to access or 
manipulate that information, then few 
organizations will take LDAP seriously. 
Programmers and administrators, in par- 
ticular, need to be able to access this in- 
formation through a variety of methods 
and tools. 

From a programmer’s perspective there 
should be several ways to access the in- 
formation. A low-level C API for directo- 
ry access, for those times when speed is 
important, needs to be embedded in a 
compiled application. Java is becoming 
more important in the corporate environ- 
ment, and as a result access to LDAP from 
Java has become a necessity. But there 
are also times when the ability to gener- 
ate a quick prototype in a scripting lan- 
guage can make the difference in a pro- 
ject’s success. 

Administrators also need several ways 
to get to the information in a directory. 
A GUI for adding and changing informa- 
tion easily is an absolute requirement. 
Command-line utilities are frequently im- 
portant for doing batch updates. Howey- 
er, sometimes command-line utilities don’t 
allow a fine enough degree of control, so 
once again access through a scripting lan- 
guage is important. 


Troy is a systems engineer for Intraware 


and can be contacted at troy@ 
intraware.com. 


72 


To address issues such as these, Netscape 
has released PerLDAP, which provides a 
mechanism for accessing directory infor- 
mation from Perl. This is an important tool 
for both programmers and administrators. 
In this article, Pll provide both a high-level 
overview of what PerLDAP does and a de- 
tailed explanation of how you can use it. 


What is PerLDAP? 
PerLDAP (available in source code form 
at http://www.mozilla.org/directory/ 
perldap.html) is a set of Perl functions 
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and objects that simplify access to LDAP 
services. Although Netscape released 
PerLDAP with the intent that it be used 
with the Netscape Directory Server, it 
should work equally well with most LDAP 
v3-compliant directories. 

Netscape now has three ways for you 
to access LDAP. The first of these is the C 
API, which is distributed in the form of 
the Directory Server SDK. This SDK is 
available for a wide variety of platforms, 
and would be used by anyone develop- 
ing an LDAP application in either C or 
C++. For Java developers, Netscape has 
the Java LDAP SDK. As you’d expect, the 
Java SDK has an object-oriented approach, 





and as such is much easier to use than 
the C API. PerLDAP also has an object- 
oriented approach to LDAP. PerLDAP is 
the most approachable of these three tools 
for accessing the Directory Server. All three 
tools (and some others as well) are avail- 
able at Netscape’s DevEdge Online (http:// 
developer.netscape.com/program/ 
home.html). 

Experienced Perl developers may already 
be familiar with the Net::LDAP- api pack- 
age available on the CPAN, the Compre- 
hensive Perl Archive Network (http://www 
.cpan.org/). Net::LDAPapi does have a Perl 
object-oriented interface, but does not pro- 
vide a general-purpose LDAP object. In oth- 
er words, Net::LDAPapi really only gives 
you access to the LDAP API without pro- 
viding a simpler-to-use object-oriented 
mechanism. PerLDAP, on the other hand, 
gives you a more general-purpose LDAP 
object that behaves much like the Java LDAP 
classes, which are part of the Java LDAP 
SDK 3.0. Both PerLDAP and Net::LDAPapi 
require the Directory Server SDK 3.0. 

PerLDAP comes with a set of Perl func- 
tions that mirror the functions in the Direc- 
tory Server SDK. These functions let you 
connect to a Directory Server, create, mod- 
ify, and delete LDAP entries. More impor- 
tantly, PerLDAP includes two objects that 
wrap around these functions, creating a 
more friendly developer experience. These 
objects are Mozilla::LDAP::Conn and 
Mozilla::LDAP::Entry. 

Mozilla::LDAP::Conn is a general- 
purpose LDAP object that is instantiated 
by calling the new() method with the ap- 
propriate parameters. The parameters are 
the host name of the directory server, the 
bind distinguished name (dn), the bind 
password, and the LDAP search base. 
Once a connection has been established, 
Mozilla::LDAP::Conn has methods for 
searching the directory server, adding en- 
tries, modifying entries, and deleting entries. 

Mozilla::LDAP::Entry is an object that 
gives you access to the components of an 
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(continued from page 72) 

LDAP entry. It also provides methods for 
modifying an entry. Many Mozilla::LDAP:: 
Conn methods return Mozilla::LDAP::Entry 
objects. 

Because most LDAP entries are in tex- 
tual format, Perl’s strong string-process- 
ing capabilities make it a natural fit. Fur- 
thermore, the ease of using the PerLDAP 
objects can make it an attractive alterna- 
tive to using the command-line tools such 
as ldapsearch and ldapmodify. PerLDAP 
gives you the ability to create an LDIF ex- 
port without shutting down the Directo- 
ry Server. It includes sample code for syn- 
chronization with PeopleSoft. It can also 
be used to batch import users from oth- 
er user databases. Only need and imagi- 
nation limit the list of possible uses. 


An Example Application 

It is common for an organization to set 
up a variety of mailing lists. The uses range 
from being an alias for an entire department 
to being a tool for collaboration. If you’ve 
ever seen an e-mail address such as “engi- 
neering-newengland@company.com,” then 
you've seen mailing lists in action. Col- 
laborative mailing lists frequently allow 
people to subscribe and unsubscribe to 
the mailing list on their own without the 
intervention of an administrator. This is 
usually done using a list server such as 
Majordomo. 

The Netscape Messaging Server sup- 
ports mailing lists, but doesn’t provide a 
means for users to subscribe or unsub- 
scribe to those mailing lists. It requires that 
administrators manually add users to 
groups. This generally encourages people 
to look to an external program such as 
Majordomo to manage their mailing lists. 

The example I present here, which 
demonstrates the inner workings of 
PerLDAP, is a CGI program that can be 
used to manage mailing lists via Direc- 
tory and Messaging Servers. I will demon- 
strate some techniques for connecting (or 
binding) to the Directory Server, pro- 
cessing the results of a search, and mod- 
ifying LDAP entries. 


Binding to the Directory Server 

Using PerLDAP 

The demo CGI program needs to do a few 
basic things, such as prompt users to log 
in, display a list of mailing groups avail- 
able on the server, and let users modify 
membership in those mailing groups. List- 
ing One is the core of this program. In a 
nutshell, it checks the state of the appli- 
cation and sends back the appropriate 
screen. The basic approach that I’ve tak- 
en with this CGI program is to separate 
the functions that display HTML from those 
that process LDAP entries. This is a good 
CGI programming practice, and it also 


makes it easier to focus the discussion on 
functions that make use of PerLDAP. 

Since users are going to be modifying 
LDAP entries, it is necessary to make them 
bind to the Directory Server. However, 
users should be able to simply type in their 
user ID and password without having to 
remember their entire distinguished name. 
Users are used to this sort of behavior, and 
you want to mimic it. It turns out that this 
is actually pretty easy to accomplish. 

There are three steps to binding to the 
Directory Server starting with just a user 
ID. The first step is to bind anonymous- 
ly to the Directory Server. The second step 
is to search for the distinguished name for 
that user ID. Listing Two shows the 
Idap_get_user_info() function that does 
these first two steps. This function also re- 
turns the full name of the user, but this is 
only for display purposes and not a nec- 
essary component to binding to the Di- 
rectory Server. The third step is to use 
the distinguished name returned by 
Idap_get_user_info() to bind to the Direc- 
tory Server. This is a common approach to 
solving this problem. In fact, the Netscape 
Enterprise Server uses a similar mechanism 
for logging into the web server using just a 
user ID and password. 

The /dap_get_user_info() function shows 
the basics of using the Mozilla:.IDAP::Conn 
package. Calling the new() method with- 
out specifying a bind dn creates an anony- 
mous connection to the Directory Server. 
Once a connection has been established, a 
search can be conducted. The search() 
method is similar to using the LDAP 
search feature in Communicator or in the 
command-line tools. You simply specify 
a search base and an attribute to search 
on. The sub parameter tells the search() 
method to look in subtrees. From there, 
it’s just a matter of processing the result 
set that search() returned. 

The search() function returns a Mozilla 
::LDAP.:Entry if the search was successful. 
You can then use the Mozilla::LDAP.:Entry 
methods to access the data in the entry. 
One such method is getDNC), which returns 
the distinguished name of that entry. An- 
other useful method is exists(), which lets 
you check if a value exists for the specified 
attribute. 

Extracting the values of attributes can be 
a little tricky. The most important thing to 
remember is that an entry can have more 
than one value for each attribute. In fact, 
it is quite common. For example, a typical 
entry for an individual will have the values 
“top,” “person,” “organizationalperson,” and 
“inetorgperson” for the objectclass at- 
tribute. This is important because when 
you ask Mozilla::LDAP::Entry for the val- 
ue of an attribute, it will return a refer- 
ence to an array. The general syntax looks 
something like: 
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$value = $entry->{attribute}larray_index]; 

You can use the size() method to deter- 
mine how many elements are in the array. 
For the purposes of /dap_get_user_infoQ), 
you know that there is generally only one 
common name per entry, so you just grab 
the first one after testing to make sure a val- 
ue exists. 

Since most searches will return more than 
one entry, a method is needed to iterate 
through the result set. That method is next- 
Entry(), which is a Mozilla::LDAP::Conn 
method. In the case of Idap_get_user_infoQ), 
getting more than one entry back indicates 
there is a problem with the directory infor- 
mation tree because Netscape requires that 
uids be unique for the entire directory. The 
last thing that needs to be done with any 
basic LDAP connection is to close it. To do 
this, simply call close(). 


Processing Search Results 

Once you know the bind dn of the user 
in question, you can bind to the Directo- 
ry Server as that user and search for a list 
of available mailing lists. Listing Three 
shows the /dap_get_mail_groups() func- 
tion, which does exactly that. This func- 
tion is a straightforward LDAP search. The 
primary differences between this function 
and /dap_get_user_info() are that you 
aren’t binding anonymously and the func- 
tion populates an array with LDAP entries 
that are the results of the search. After call- 
ing /dap_get_mail_groups(), you have an 
array full of Mozilla::LDAP::Entry objects. 

Listing Four shows what you can do 
with that array, although this is not the 
only way to process the results of a search. 
Rather than stuffing each entry into an ar- 
ray, the /dap_get_mail_groups() function 
could have processed each entry as it was 
retrieved. However, I wanted to be able 
to reuse my primary search so I separat- 
ed it into an another function. 

Listing Four demonstrates how to look 
through each entry to see if the user is a 
member of that group. The group lists 
members by using the uniqgueMember at- 
tribute. In other words, to find a user in 
a group, you have to step through the ar- 
ray of uniqueMember attributes looking 
for the member. This is done by finding 
out how many elements are in the array 
using the size() method. Once this is 
known, it’s simply a matter of using a for 
loop to look at each entry. You exit the 
for loop when you find the member. 


Modifying Entries 

The last major piece of functionality in 
PerLDAP that I’ll examine is how to mod- 
ify an entry, specifically how to add or 
remove an attribute and its correspond- 
ing value. The process is similar for both 
actions, so I'll take a look at adding an 
attribute. Listing Five shows how to do 
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this. First, connect to the LDAP server. 
Second, find the entry that you want to 
modify. Third, add the attribute value. 
This is done with the addValue() method 
of the Mozilla::LDAP::Entry object. Finally, 
update the entry on the server. The update() 
method of the Mozilla::LDAP::Conn object 
has a single parameter, which is the Entry 
that has been modified. This is an impor- 
tant step. If wpdate() isn’t called, then the 
Entry is only modified in memory and not 
on the server. 


Making the Mailing List Manager 

If you’ve spent any time poking around 
the access control mechanism for the Di- 
rectory Server, you may have noticed a 
permission called “selfwrite.” Don’t be 
surprised if Netscape renames this per- 
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mission in a future release because it 
doesn’t do what you’d expect. The first 
thing that comes to mind is that setting 
this permission for a group of users 
would allow them to modify their own 
entry. However, what it really does is al- 
low users to add or remove themselves 
from a groupofuniquenames, which is 
also a mailGroup. 

If you are using the Netscape Messag- 
ing Server in conjunction with the Direc- 
tory Server, the installer for the Messag- 
ing Server extends the default schema to 
support the use of groups as mailing lists. 
In other words, if a group is given an e- 
mail address, then the Messaging Server 
will send an e-mail addressed to all mem- 
bers of the group. A typical use of this 
functionality would be to create e-mail 
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aliases for departments or teams. The self- 
write permission allows users to modify 
their membership in the group. 

With a little creativity, this feature can 
also be used to mimic some of the func- 
tionality of a list server such as Major- 
domo. An important feature of Major- 
domo lets users self-administer their 
membership in mailing groups by using 
commands sent by e-mail. PerLDAP 
makes it easy to create a web-based self- 
administration tool. Listing One shows the 
basic framework of such an application. 
The process_login() function makes use 
of binding to the Directory Server, search- 
ing, and processing the result set. The 
process_submit() function also adds or 
removes attributes from Entries. These 
functions both call functions that display 
HTML as well. The complete source code 
is available electronically from DDJ (see 
“Resource Center,” page 5) or by writing 
me at troy@intraware.com. 

You must grant the selfwrite privilege 
to at least some groups and users for the 
Mailing Group administration CGI to work. 
Refer to the Directory Server Administra- 
tor’s Guide if you need help in setting up 
access control on the Directory Server. Be 
sure to look at the comments at the be- 
ginning of the source code because they 
will tell you what all the necessary com- 
ponents are and how to point the pro- 
gram at your Directory Server. 

There are a couple of other things about 
this program you should be aware of. The 
first is that the program passes the user’s 
password as a parameter and as a hidden 
field. This means that if you use it in an 
extranet environment you should be sure 
to use SSL so that this information is en- 
crypted. Second, you don’t want to tread 
lightly in the area of access control. If you 
are working with a production Directory 
Server, be sure to coordinate all activities 
with the primary administrator because it 
is easy to lock everyone out. 


Conclusion 

I've presented just one possible use for PerL- 
DAP. There are many ways this tool can be 
used to simplify Directory Server adminis- 
tration. The key things to remember about 
doing a search with PerLDAP are: 


e Set up the connection using new(). 
e Make the search using search(). 

e Process the results. 

¢ Close the connection. 


There are, of course, many variations 
on this theme, but if you keep these four 
steps in mind it should be relatively easy 
for even beginning Perl developers to 
make good use of this tool. 


DDJ 
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Listing One 

unless (Scgi->param()) { 
html_display_login() ; 

} elsif (Scgi- param ("exit") eq "Exit") { 
html_display_login() ; 

} elsif ($cgi->param("mode") eq "login") { 
process_login() ; 

} elsif (Scgi->param("mode") eq "submit") { 
process_submit () ; 


e e 
Listing Two 
sub ldap_get_user_info { 
my Suser = $_[@]; # The user ID 


my Sbind_dn; 
my Scn; 
my Sanon_conn = new Mozilla::LDAP::Conn($ldap_host, 
Sldap_port, "", "", "") || die "Can't connect to $ldap_host.\n"; 
my Sentry = Sanon_conn- * >search($search_ base, "sub", "(uid=Suser)"); 


f (! Sentry) { 
return; 
} else { 
my Si = @; 
while(Sentry) { 
Sit+; 
Sbind_dn = Sentry->getDN() ; 
if (Sentry->exists("cn")) { 
Sen = Sentry->{"cn"}[@] ; 
} else { 
return; 


Sentry = $anon_conn->nextEntry() ; 


} 

$6453 > 4) 4 
return; 

} 


} 


Sanon_conn->close(); 
return (Sbind_dn, S$cn); 


e e 
Listing Three 
sub ldap_get_mail_groups { 
my Sbind_dn = Sa: # The user's bind DN 


my Spassword = $_[1]; # The user's password 

my Smgref = $_[2]; # A reference to an array of Entry objects 

my $conn = new Mozilla::LDAP::Conn($ldap_host, $ldap_port, $bind_dn, 

Spassword, ""); 

my Sentry; 

my Scn; 

if (! $conn) { 
html_display_error( "Couldn't connect to the directory server."); 
exit (@); 

} else { 
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Sentry = S$conn->search(S$search_base, "sub", "objectclass=mailGroup") ; 
if (! Sentry) { 
html_display_error("There aren't any mailing lists on this 
server."); 
exit (Q) ; 


Mh le(jentty) { 
Sen = Sentry->{"cn"} [0] ; 
f ads the entry to the array unless it's the Postmaster group 
f ($cn ne "Postmaster") { 
push (@Smgref,Sentry) ; 


Sentry = $conn->nextEntry(); 


Sconn->close(); 


} 
J 


Listing Four 
foreach (@mailGroup) { 
SgroupName = $_->{"cn"}[@]; 
Sdescription = $_ ->{"description"} [0 1; 
my SisMember = Q; 
my Sct = $_ ->size("uniquemember") ; 
for ($i = @: Si < Sct ; Sit+) { 
if ($bind_dn eq $_->{"uniquemember"} [$i]) { 
SisMember = 1; 
last; 


} 


} 
html_row(S$isMember, SgroupName, $ct, $description) ; 


at e e 
Listing Five 
sub ldap_add_to_group { 
my Sbind_dn = $_[@]; i The user's bind DN 
my jposeucrd = $_[1]; 4 The user's password 
my $mg_cn = $_[2]; # The CN of the Group 
my Sconn = new Mozilla: :LDAP: :Conn($ldap_host, $ldap_port, S$bind_dn, 
Spassword, nny ; 
if (!Sconn) { 
html_display_error( "Couldn't connect to the directory server."); 
exit (Q) ; 
} else { 
my Sentry = $conn->search($search_base, "sub", "(cn=Smg_cn)") ; 
if (!Sentry) { 
html_display_error( "Couldn't find $mg_cn.") ; 
exit (@); 
} 
Sentry->addValue("uniquemember", $bind_dn) ; 
Sconn->update (Sentry) ; 
Sconn->close(); 
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PROGRAMMER’S TOOLCHEST 





f you are like me, you like writing tests 

for your code about as much as you 

like writing end-user documentation. 

Typically, you might write a separate 
application or module to test a module in 
your software application. This type of 
testing (called “unit-level testing”) is usu- 
ally given short shrift — if it is included in 
the project plan at all. I don’t like writing 
unit-level tests in most modern structured 
programming languages. Since the test 
code is usually more complicated and 
harder to write than it should be, I end 
up not testing nearly as much as I should. 
Intellectually, I know testing is good, but 
time spent away from coding the main ap- 
plication seems hard to justify emotional- 
ly. Then I stumbled upon JPython. While 
it may not be the ultimate answer, JPython 
can help you quickly produce unit tests 
for your Java packages. 

JPython is a freely available version of 
Python (http://www.python.org) imple- 
mented in 100 percent pure Java. Python, 
a cross-platform programming language 
implemented on most major platforms, 
can be used for a variety of purposes. 
Python’s abilities as a scripting language 
are important when it comes to writing 
test applications. Features that make 
Python a good scripting language include: 


e No separate compile and link phase. 

e No type declarations. 

e Dynamic loading and reloading of 
modules. 

e Huge library of add-on modules. 


Kirby is a senior consultant with Dataline 
Inc. He is also a Microsoft Certified Solu- 
tion Developer, Charter Member. He can 
be reached at kirbyangell@hotmail.com. 
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Examining JPython 


A Java test engine 
puts Python to the test 


Kirby Angell 


e Access to the built-in compiler and 
bytecode interpreter. 


To this list, JPython (written by Jim 
Hugunin and available at http://www 
.python.org/jpython/download) adds 
seamless access to Java packages and 
classes. (When downloading JPython, I 
recommend you also download the 
Python 1.5.2 library. Part of Python’s 
power comes from its extensive library 
of add-on modules. The standard 
JPython distribution comes with only a 
subset of the Python library.) Since 
JPython is implemented in Java and runs 
in the Java Virtual Machine, using Java 
classes in Python code is trivial. As a 
scripting language with access to Java 
classes, JPython lets you quickly whip 
up a test of any Java class. 

There are two issues I’ve run across 
concerning portability and JPython. First, 
JPython does not seem to work with the 
Kaffe JVM distributed with Red Hat Lin- 
ux (http://www.redhat.com/). However, 
it does work fine with the Blackdown JVM 
for Linux (http://www.blackdown.org). 
Second, because JPython does on-the-fly 
bytecode loading, some Just-In-Time GIT) 
compilers can have problems. Still, I’ve 
written some pretty advanced code with 
JPython and have only had to turn off the 
JIT compiler once. 


JPython Console Interface 

Since JPython includes a console inter- 
face, you can begin using it without writ- 
ing any code. When you start JPython 
Without any command-line options, you'll 
see JPython’s startup message, as with Ex- 
ample 1(a), which uses the JVM shipped 
with JBuilder2. The “>>>” is JPython’s 


command prompt. From there you can 
begin typing any valid JPython command. 
I use this console interface extensively to 
explore the functionality of unfamiliar Java 
classes. By exploring their inner workings, 
I save time in the edit, compile, and test 
phases of Java coding. Let’s say I just 
learned that the java.math.BigInteger class 
might solve my integer problems, but from 
the spartan documentation, I haven't fig- 
ured out how to use it. To explore this 
class in JPython, I must first make JPython 
aware of its existence using Example 1(b). 
This is actually the same syntax you would 
use to import a regular Python class. Next, 
I can try out one of the constructors for 
BigInteger and view the results; see Ex- 
ample 1(c). 

The first line creates a new instance of 
the BigInteger class. As you’d expect, 
JPython looks for an appropriate con- 
structor of BigInteger and creates a new 
object (61). When displaying the contents 
of bi, JPython also looks for an appropri- 
ate way to display the object, by eventu- 
ally calling the foString method of the ob- 
ject. If JPython cannot find an appropriate 
constructor, then it would report an ap- 
propriate error. If you wanted to test Big- 
Integer’s max method, you could try Ex- 
ample 1(d). 

There is one JPython feature that 
makes scripts that use Java cleaner look- 
ing and sometimes easier to write. 
JPython uses the JavaBean Introspector 
to identify class properties either from 
common design patterns (methods named 
getX or setX) or from explicitly specified 
BeanInfo. Although illegal in Java, Ex- 
ample 2(a) lets you write this code us- 
ing java.util.Date’s getHours and setHours 
methods. 


Dr. Dobb’s Journal, April 1999 








————_ 





Secure Your A 





Don't let the threat of a few pesty hackers ruin your 
day. Build your applications with RSA Data Security 
components and you'll build in security that stops 
hackers dead in their tracks. 


That's why RSA, a Security Dynamics’ Company, is the 
crypto-security source for more than 400 leading 
OEM companies. And why you'll find it in more than 
300 million active installations worldwide, ranging 
from smart cards and cell phones to Web browsers 
and e-commerce systems. 


RSA makes security a real picnic. Since 1982, RSA 
security technology has been the foundation of the 
information industry. Today, we offer a complete set 
of security components that allow you to create 
secure C and Java programs as well as SSL and 
S/MIME enabled applications. 





As a developer working with RSA, you simply select 
the library components you need and build them 
into your application. And you can be sure RSA 
security will keep swarming hackers at bay. 


Send the world (and its hackers) a message. 
The RSA name is proof to you and to users that 
your application will deliver information cleanly 
and securely. To learn more about RSA's complete 
package of data security solutions, please visit 

our website at www.rsa.com/developersolutions. 
And start developing some killer apps of your own. 
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(continued from page 78) 

Unfortunately, I don’t recommend us- 
ing this inside of your own JPython test 
scripts. There is nothing wrong with the 
functionality — it works fine— but I like 
being able to use my test scripts as sam- 
ple code for the consumers of my Java 
classes. Other developers reading my 
scripts really need to see the getX and setx 
methods that they would have to use in 
their own Java programs. This example 
without the Bean properties looks like Ex- 
ample 2(b). It’s not as pretty, but makes 
for a better example of how to use the 
Date class. 


JPython Test Script 
Listing One is a JPython program to test 
the java.math BigInteger class. When you 


Still reinventing? 


examine Listing One, you will see more 
involved tests of Big/nteger than those I 
just described, plus a new test for Big- 
Integers abs method. The structure of this 
program is typical of the test scripts I first 
created with JPython: 


1. Print an introductory message (line 6). 

2. Perform one or more tests (lines 7-22). 

3. Print a closing message saying all is well 
(line 23). 


Python, unlike some scripting lan- 
guages, is a “real” language and supports 
procedures, classes, and inheritance. I cre- 
ated a base class (TestCase) designed to 
make the quick and consistent coding of 
tests possible. Listing Two performs the 
same tests as Listing One, but within a 
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Python class derived from TestCase (shown 
in Listing Three). Although the Big/nte- 
gerTest class seems to be a bit longer than 
our first version in Listing One, it contains 
more functionality, and each actual test is 
shorter. Since each test is segregated into 
its own class method, you can easily iden- 
tify the individual tests and add new tests 
without worrying about side effects from 
other tests. 


TestCase 

Before considering all of the intricacies of 
Python classes and the TestCase base class, 
look at the code involved in writing a sin- 
gle test. Compare lines 7-11 of Listing One 
and lines 9-12 of Listing Two. In Listing 
One, the code performs an operation, 
checks the result, and if the result wasn’t 





Example 1: (a) JPython’s startup 
message; (b) making JPython aware of 
the existence of a class; (c) trying out 
(a) of the constructors; (d) testing 
BigInteger’s max method. 





Example 2: (a) Writing code using 
java.util.Date’s getHours and 
setHours methods; (b) a better 
example of using the Date class. 
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what was expected, prints an error and 
exits. My problem with Listing One is that 
it takes some mental effort to construct 
the test, and this effort is duplicated in 
each test. The corresponding code in List- 
ing Two replaces the /f statement and the 
call to sys.exit(D with assert_. This method 
is defined in the TestCase base class. Like 
the standard assert functions found in oth- 
er languages, TestCase’s method only 
jumps into action if the statement passed 
to it evaluates to False (0 in Python). The 
single call to assert_ replaces the Jf, print 
error, and exit functions in each test of 
the original code. 

You should also notice that what was 
a comment in Listing One (# Test String 
Conversion) is now in a special quoted 
string called a “docstring.” In Python, 
practically everything is an object, in- 
cluding the functions, and functions have 
a __doc__ property that returns this 
quoted string. Since the docstring can be 
accessed programmatically, the test har- 
ness can now display information about 
the test. In effect, a line in the test script 
that was useful only to the developer of 
the test is now doing double duty when 
the test is run. In TestCase, the docstring 
is printed to the console before the test 
is run, providing onscreen commentary 
for free. 


To run the new TestCase-derived unit 
test for Biginteger, simply run JPython with 
the name of the script on the command 
line. Example 3 shows the results on my 
computer. 

When JPython loads BigIntegerTest.py 
(available electronically; see “Resource 
Center,” page 5), it immediately executes 
lines 23-27, which eventually call Test- 
Case’s runTest method. This method is 





responsible for executing the test meth- 
ods within the class. In addition, this 
method is responsible for creating the 
output just mentioned. 

runtTest is the main workhorse of 
TestCase and any of the derived class- 
es. Using Python’s inspection capabil- 
ities, runTest first obtains a dictionary 
of all the methods and properties avail- 
able in the class. If a method begins 








Example 4: (a) Dynamically loading test modules; (b) example of an addTests 


method for the BigIntegerTest.py module. 
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flickerless updates. 
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with “test_,” runTest performs the fol- 
lowing activities: 


e Calls the class initialize method. 
e Writes out the docstring. 

e Executes the method. 

e Calls the reset method. 


When there are no more methods, 
runtTest exits. Since the initialize and re- 
set methods are executed before and af- 
ter each test, they can hold code common 
to all of the tests— rather like a function- 
level constructor and destructor. These 
methods also serve to ensure that each 
test is run in an environment separate from 
the other tests. This, together with each 
test being in a separate method, cuts down 
on side effects between tests. 

Remember that TestCase’s assert_ 
method is responsible for reporting any 
problems in the test. Its main code is ac- 
tivated only if the expression passed is 
False. If the expression is False, the as- 
sert_ method outputs the name of the file 
and line number where the assertion failed 
and calls the writeError method to display 
the error message. A raise call at the end 
of assert_ sends the error back to the 
caller. In the case of BigIntegerTest.py, the 
error is simply ignored and the unit test 
exits. My feeling is that once the unit has 
failed one test, the rest of the tests are sus- 
pect. Once a test fails I do not bother run- 
ning any remaining tests on that unit. 

TestCase contains one other method 
that I have not discussed. failTest keeps 
track of whether a test in this unit test 
has failed. If the unit test exits immedi- 
ately after a failure, why keep track of 
the failed test? 


Batch Testing 

Once you have imposed a modicum of 
structure on the unit-level tests, it is not 
a far stretch to implement all sorts of au- 
tomated, batch, and regression testing. The 
testsuite.py and testeng.py scripts (both 
available electronically) implement batch 
testing, but they can easily be extended 
to support all manner of testing, while still 
allowing simple test cases to be written 
quickly. 

Testsuite.py contains a single class 
called TestSuite that does not really do 
much besides act as a collection class for 
the TestCase-derived class. TestSuite has 
the typical collection methods —add, re- 
move, and count—to manage the con- 
tents of the collection. The interesting 
methods in JestSuite are run, and runTest 
actually executes all of the tests in the 
suite and a single test, respectively. The 
run method simply iterates through the 
collection and calls the runTest method 
for each test. runTest performs the same 
function that the “__main__” code did 
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in Listing Two. It calls TestCase’s runTest 
method and intercepts any exceptions 
generated by the test. 

Testeng.py is the master integrator be- 
tween TestSuite, TestCase, and the unit-lev- 
el tests you design. Testeng.py loads each 
test class into a TestSuite, then executes 
the TestSuite by calling the run method. It 
accomplishes this magic using two cool 
features of JPython (and Python for that 
matter). First, JPython lets you dynamical- 
ly build, compile, and execute Python code 
on-the-fly. Second, JPython provides func- 
tionality to dynamically load new modules. 
Near the end of testeng.py, Example 4(a) 
dynamically loads each test module. 

The module is imported into JPython 
and a handle to that module is retrieved 
in the first two lines. If the module con- 
tains a procedure called addTests, that 
method is called with a TestSuite object, 


Testeng.py will attempt to load any .py 
file in the current directory that does not 
begin with “test” (so testcase.py, test- JPython classes can subclass Java classes. 


JPython also makes 
an excellent tool 


for prototyping 
Java applets 





suite.py, and testeng.py will not be mis- 
taken for actual tests). 


ages in a Java application and use JPython 
as your application’s scripting engine. 


JPython also makes an excellent tool for 
prototyping Java applets that are embed- 
ded in a web browser. 

Still, the testing framework described 
here is not dependent on JPython. The 
code works unmodified under the DOS, 
Windows, and UNIX versions of Python. 
If your bailiwick is Windows and COM, 
the Python “win32all” add-in lets you 
test COM components the same way 
JPython lets you test Java packages and 
classes. 

The test harness included with this ar- 
ticle is basic in nature, but can be ex- 
tended. Its primary focus is to enable you 
to create unit tests with as little work as 
possible. In my experience, it’s best to 
write test cases quickly, or you won't write 





allowing each module to control how tests them at all. 
. e 
are added to the test suite. Example 4(b) Conclusion 
is an example of an addTests method for Since JPython is written in Java, it is 
the BigIntegerTest.py module. straightforward to include JPython pack- DDJ 
ist 95: from string import * 
Listing One 96: class TestCase: 
Qi: # 97: "Base class for python based unit level and regression tests. \ 
02: # BigIntegerTest.py 08: To use this class, implement the runTest, initialize, and reset \ 
@3: # Tests the standard java class java.math.BigInteger 9: methods. Call the assert_ method to test for success." 
@4: import sys 19: def __init__(self, name): 
@5: from java.math import BigInteger 1 self.testName = name 
@6: print "Test the standard java BigInteger class." 12: self.failed = 1 
@7: # Test string conversion 13; self.failureReason = "<<Not Tested>>" 
@8: bi = BigInteger( '100' ) 14: def runTest(self): 
@9: if bi.longValue() != 100: 15: "Run the test methods in this class" 
10: print "BigInteger string conversion failed." 16: names = dir( self.__class__ ) 
11: sys.exit(1) if: self.failed = @ 
12: # Test max operator 18: self.failureReason = "" 
13: bil = BigInteger( '106' ) 19: for name in names: 
14: bi2 = BigInteger( '200' ) 20: # if this is a test method 
15: if bil.max(bi2) != 200: zs if name[:5] == "test_": 
16: print "BigInteger max operator failed." 223 self.initialize() 
17: sys.exit(1) re try: 
18: # Test absolute value operator 24: func = getattr( self, name ) 
19: bi = BigInteger( '-100' ) 25: self.writeMessage( func.__doc__ ) 
26: if bi.abs() != 100: 26: # Execute the test 
21: print "BigInteger abs operator failed." re func () 
22: sys.exit(1) 28: finally: 
23: print "All tests completed." 29: self.reset() 
30: def initialize(self): 
313 "Called before each test is run. Should be overridden.": 
— 32: pass 
33: def reset(self): 
Listing Two 34: "Called after each test is run. Should be overridden." 
O1: # 203 pass 
02: # BigIntegerTest.py 36: def failTest(self, reason): 
@3: # Tests the standard java class java.math. BigInteger 37: "Mark this test as having failed." 
@4: import sys 38: self.failed = 1 
@5: from java.math import BigInteger 39: self.failureReason = reason 
@6: from testcase import TestCase 4G: def writeMessage(self, message): 
@7: class BigIntegerTest (TestCase) : Al: "Writes a message to the standard output device." 
08: "Test the standard java BigInteger class." 42: self.__writeMessage(sys.stdout, message ) 
09: def test_stringConversion(self): 43: def writeError(self, error): 
10: "Test string conversion" 44; "Writes an error to the standard output device." 
11: bi = BigInteger( '100' ) 45: self.__writeMessage( sys.stdout, "ERR " + error ) 
‘2 self.assert_( bi.longValue() == 100 ) 46: def writeWarning(self, warning): 
13: def test_maxOperator (self): 47: "Writes an warning to the standard output device." 
14: "Test max operator" 48: self.__writeMessage( sys.stdout, "WRN " + warning ) 
15: bil = BigInteger( '100' ) 49: def __writeMessage(self, outFile, message): — 
16: bi2 = BigInteger( '200' ) 50: year, month, day, hour, minute, second, weekday, julian,\ 
17: self.assert_( bil.max(bi2) == 2@@ ) 51: daylight = localtime(time() ) 
18: def test_abs(self): 52: outFile.write( "%@4d%O2d%O2d %O2d:%O2d:%O2d\t" % (year, \ 
19: "Test absolute value operator" 533 month, day, hour, minute, second) ) 
0: bi = BigInteger( '-190' ) 5a: outFile.write( "%s\t%s\n" % (self.testName, message) ) 
21: self.assert_( bi.abs() == 100 ) 55: def assert_(self, test): 
22: if _name _ == "__main__": 560: "Logs and generates an error if the test is false." 
23: try: oy ie if test == 
24: c = BigIntegerTest( "BigIntegerTest" ) 58: stack = traceback.extract_stack() 
25: c.runTest () 59: stackFrame = stack[len(stack) -2] 
26: except: 60: file,line,method,call = stackFrame 
27: pass 61: path,file = os.path.split (file) 
62: source = "%s %s %s %s" % (file, line, method, call) 
63: self.failTest( "Assertion failed: %s" % source ) 
64: self.writeError( self.failureReason ) 
Listing Three 65: raise AssertionError, source 


01: import sys 

@2: import traceback 
03: import os 

@4: from time import * 
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PROGRAMMER’S TOOLCHEST 





hen Microsoft released Windows 
3.1, UNIX developers started get- 
ting demands for their software 
on this new platform. Since the 
advent of Windows NT 4.0 and the ex- 
plosion of TCP/IP and the Internet onto 
the non-UNIX desktop, thousands of pro- 
grams have been ported to Windows. Ear- 
ly ports were often only a glorified re- 
compile decorated with an after-the-fact 
GUI that replaced the command-line in- 
terface. 

In contrast, today’s major cross-platform 
tools are just as likely to have started out 
on Windows as on UNIX. Software de- 
velopers are finding that customers in the 
midst of migrating from Solaris or HP to 
NT still want UNIX clients Gf not full- 
blown applications) that run on their old 
boxes. Thus, more and more program- 
mers are doing something that once felt 
inherently backward Gf not evil)— port- 
ing from Windows to UNIX. 

For large C++ programs, the most re- 
alistic solution for porting from Windows 
to UNIX is to use a tool that implements 
the Windows API natively on the target 
system. One such product is MainWin 
XDE (eXtended Development Environ- 
ment) 3.1 from MainSoft (http://www 
-mainsoft.com/). The company I work for 
recently finished porting a major appli- 
cation from Windows NT to Solaris, us- 
ing MainWin with satisfactory results. 
MainWin provides MFC Libraries, com- 
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mon controls, a resource compiler, help 
engine, automatic makefile generator, spe- 
cialized source files to include with ap- 
plication programs, and a whole set of tools 
that implement the cross-development 
environment on target systems. The ad- 
vantages provided by using MainWin in- 
clude a common code base, a relatively 
easy porting step with regards to compi- 
lation and linking, and an application that 
executes native code on the target plat- 
form. You can also offer either a Win- 
dows or “motif” look to your programs. 
The principal disadvantage of MainWin 
is a sometimes sluggish (but certainly not 
intractable) run-time performance with 
regard to graphics. 

In this article, I'll discuss the specifics 
of using MainWin to port an executable 
application that compiles and runs in Win- 
dows to UNIX. I'll discuss preparing the 
Windows code and planning the port to 
UNIX, preparing the UNIX environment, 
dealing with third-party libraries, gener- 
ating MainWin makefiles, building the pro- 
ject, and several run-time hurdles that you 
might have to clear. Along the way, I'll 
point out stumbling blocks that repeated- 
ly caused us headaches and some inter- 
esting technical hurdles that MainSoft had 
to handle to provide its relatively seam- 
less integration. 


The MainWin Environment 

Currently, MainWin is supported for 
SPARC Solaris 2.5.1, HP-UX 10.20, and SGI 
Irix 6.2 through Irix 6.5. You'll also need 
Native X11 Release R5 or RO on all sup- 
ported platforms. MainWin runs on top 


Porting C++ Code 
trom NT to UNIX 


Using the MainWin XDE toolkit 


of X Windows interacting with both the 
the X Server and the window manager. 
Many window managers are supported, 
including twm, olwm, mwm, vuewm, 
4Dwm, and olvwm. For most uses of 
MainWin a few environment variables 
need to be set. MWHOME refers to the 
root directory of your MainWin installa- 
tion. MWREGISTRY stores the location of 
your registry file. Depending on your OS, 
the run-time libraries, MFC libraries, set- 
up scripts, and other features are stored 
in logically organized directories under 
the installation root. One of the first things 
youll notice using the tool is that it’s much 
easier to find things with this hierarchical 
organization than in Windows, where 
most applications, DLLs, and help files 
are dumped in two main and several oth- 
er lesser installation directories. 

An important issue in planning any 
cross-platform development effort is the 
basic problem of how to access source 
code on multiple platforms. In my case 
I used ClearCase, Rational’s multiplat- 
form code-management system (http:// 
www.rational.com/). You could also use 
a shared file server. Clearly, the headaches 
caused by copying code back and forth 
between systems either via FTP or flop- 
py disks far outweigh any time that you 
might spend in setting up a proper code- 
sharing schema. 


Planning the Port to UNIX 

My first worries when faced with port- 
ing a large Windows program to UNIX 
centered around the ignoble task of plac- 
ing #ifndef WIN32 directives around the 
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(continued from page 84) 

Windows API calls and replacing them 
with UNIX and X commands. But since 
MainWin implements the Windows API, 
none of this is necessary. In fact, having 
worked on nontrivial ports before, I 
wasn’t ready for the relative luxury of 
porting using MainWin. One of the para- 
doxical tricks about this port was that 
the traditionally platform-safe practice 
of protecting OS-specific calls with pre- 
compiler directives actually backfired 
with MainWin because it defines WIN32 
and _WIN32! 

Although you don’t need to isolate the 
Win32-specific code in your project, it 
is important to verify that it compiles and 
links properly on Windows. It was our 
experience that taking the time to debug 
and test the code on Windows first saved 
considerable time on the UNIX side (here 
there were invariably several compile 
and run-time issues to deal with no mat- 
ter how meticulous I tried to be in an- 
ticipating the UNIX port). You will also 
need to obtain thoroughly tested ver- 
sions of any third-party libraries that are 
required by your Windows project. This 
step alone can cost weeks of schedule 
time if the libraries are unavailable or 
worse yet, need to be ported or coded 
from scratch. 

A MainWin project is a collection of 
MainWin modules. MainWin modules 
produce either an executable, DLL, or 
static library. If your Windows project is 
sufficiently complex, or if you are unfa- 
miliar with the layout, you might want 
to create a link dependency table to sug- 
gest the build order in the MainWin 
makefile. Figure 1 shows the directory 
structure of the simple Windows project 
TestBench (available electronically; see 
“Resource Center,” page 5), an executable 
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quickly and reliably, only OpusMake makes it! Because you've got hetter 
things to think about. 
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derived from the directory structure of 
Figure 1. Remember that run-time de- 
pendencies such as dynamic linking and 
automation can be ignored when deter- 
mining build order because they don’t 
affect the compile and link process. 


Generating a MainWin Makefile 

The only significant new code that you'll 
have to write in porting a Windows pro- 
ject to MainWin involves custom make- 
files for each Windows project. Whereas 
Microsoft insulates you from the details of 
the actual makefile syntax, MainWin re- 
quires that you delve into the nitty gritty 
of makefiles to support your applications 
on UNIX. The MainWin utility mwgen- 
make partially automates the process of 
generating makefiles, but the bulk of the 
work consists of customizing the scripts 
via a suite of MainWin defined variables 
in Figure 2. A MainWin makefile is still 
fairly simple compared to a standard UNIX 
makefile because each of your makefiles 
will in turn include one of the generic 
mainwin makefiles that do most of the 
work for you. 

The project TestBench consists of: 
TestBench.cpp, main.cpp, TestBench.rc, 
TestBench.def, and TestBench.idl. The 
first step in producing a MainWin make- 
file is to run mwgenmake in the home 
directory of TestBench (where all of the 
source files reside). Mwgenmake will put 
all C and C++ files in this directory into 
the makefile it generates. In addition, it 





will look for a module definition file (def) 
for the module name type (either .exe or 
dll), and whether the module is an MFC 
ActiveX control (.ocx). You can run mw- 
genmake from the command line or use 
the wizard version that is invoked with 
no parameters. The basic information con- 
tained in the makefile is a list of the source 
files for our application TestBench, its type 
(.exe), its name (TestBench), names of the 
Windows and UNIX resource files, com- 
pile flags (that enable you to specify the 
include file directory, a path to the .h files, 
and whether to build optimized or debug) 
and link flags (to specify a list of libraries 
to link against), and the dependencies of 
the project. 

Listing One is the actual makefile for 
our project. Of course you'll need to gen- 
erate a separate makefile for every Main- 
Win module that is part of your project. 
Also, I edited the makefile to include link 
dependencies and include paths. 


Building the MainWin Project 

By the time you get around to compil- 
ing your newly created MainWin pro- 
ject, the brunt of the work shifts from 
fitting your code into the MainWin 
schema to the inevitable fighting with 
compiler differences. A full list of the 
differences between the Microsoft Visu- 
al C++ compiler and the various UNIX 
compilers is beyond the scope and pur- 
pose of this article. Suffice it to say that 
the supported UNIX compilers deal with 


Module Type Link ——__ Run-time Bulg ~=—_ 
~~... +Dependencies§ Dependencies Order 
DLL1 DLL LiB3, LIB2 none 5 
LIB1 lib none none 1 
LIB2 lib none none 2 
LIBS lib none none 3 
ActiveX1 OCX LIB1 TestBench 4 
TestBench exe ActiveX1, DLL1 ActiveX1 6 


Table 1: Link-dependency table for TestBench. 
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Figure 2: User-configurable variables used in MainWin makefiles. 


Dr. Dobb’s Journal, April 1999 87 








O10) BY 3131 0G 
Version 1.0 


SOURCE CODE 
MODIFICATION 


CODEFIX Version 1.0 is a 
programmable tool for 
modifying all C and C++ 
source code on a file or 
project basis. 

Mi Calendar (Y2K) symbol 
identification, commenting, 
and correction. Foreign and 
domestic calendar problems 
can be found and corrected. 

@ Program Layout 
Modification. 

@ Obfuscation and shrouding 
for C++ programs. 

Insertion of code for dynamic 
runtime testing. 

@ Generating HTML 
Documentation from C++ 
programs. 

i Database generation from 
C++ source code. 


ABRAXAS" 


Software, Inc. 


Ong NO On 
Version 8.0 


PROFESSIONAL 
LANGUAGE DEVEL- 
OPMENT TOOLKIT 


Includes “Drop In” Language 
Engines for SQL, dBASE, 
POSTSCRIPT, HYPERTALK, 
SMALLTALK-80, C++, C, 
PASCAL, PROLOG, FORTRAN, 
COBOL, BASIC, SGML, ASN, 
RPG, REXX, PL1, SNA, RTF, 
VISUAL BASIC, SQL2, DB2, 
VHDL, HTML, VMRL, JAVA, 
ODMG-ODL/OQL, SQL3, MOD- 


ULA-3, DELPHI, VBS, and ADA. 


@ Portable Object Oriented 
Classes for C++ and JAVA — 
Error, Symbol Table, Syntax 
Tree, Yacc, and Lex. 

@ Debugging tools include 
runtime Visual Parser 
Debugging, Abstract Syntax 
Tree yeneration, and Cross 
Referencing. 


DOS $495, WIN 98/2000 $995, UNIX $1995 


CMI offers a broad range of consulting services 


CodeCheck “ 
AV/Sus (ee mone 


SOURCE CODE 
ANALYST . 


Includes “Drop-In” Rules 

for Compliance analysis, 
Adherence to specifications, 
Measures of complexity, Silent 
error detection, Code maintain- 
ability, and Portability. 


30 day Money back 
guarantee! Free AIR 
Shipping anywhere in 
the world! 


www.abxsoft.com 


ABRAXAS" 


Software,Inc. 
4726 S.E. Division St. 
Portland, OR 97206 


TEL (503) 232-0540 
FAX (503) 232-0543 
E-Mail: sales @ abxsoft.com 


To Order Call 


1-800-347-5214 


RATIONAL 


Ss O F T W A RF E 


Cee Education Partner 


CLEARCASE™ 
CLEAR DDTS™ 


Outsource Your 
Configuration Management 


in Software Configuration Management, 


Requirements Management, and software 
manufacture for software development 
organizations — from training 
to process outsourcing. 


88 


Configuration Management, Inc. 


1-800-550-5058 
140 Broad St, Red Bank, NJ 07701 
Tel 732-450-1100 © Fax 732-450-0715 
E-mail: ddj@softwareconfiguration.com 


www.softwareconfiguration.com 








templates differently, have varying de- 
grees of ANSI compliance, and will like- 
ly generate unexpected warnings, if not 
errors, at first. Be particularly stealthy in 
your search for problems that arise from 
differences in naming conventions be- 
tween Windows and UNIX. Hard-coded 
path strings will generally not be valid 
on a UNIX system and will cause prob- 
lems at run time. MainWin provides the 
preprocessing tool mwprepro that con- 
verts both the names of files found in 
#include directives in the source code 
and the names of files found in the cur- 
rent directory to lowercase. This was too 
violent a treatment for my taste, so I 
searched for these problems by hand 
(and not without cost!). Other possible 
prebuild steps might include rewriting 
assembly-language routines and remov- 
ing all instances of language features that 
are only supported in Visual C++, such 
as full method names within class defi- 
nitions, extraneous semicolons, remarks 
after #endif, empty arrays within struc- 
tures, nameless structures or unions, and 
the use of large integers. 

You'll be ready to make an initial stab 
at building at this point. To execute Main- 
Win’s make utility on your project, run mw- 
make in the directory of each of your Main- 
Win projects in the predetermined build 
order. The MainWin build system, through 
mwmake, will automatically invoke the 
MainWin DIP utility mwdip at link time to 
build a file that includes all necessary DLL 
initialization according to the order found 
in the link line of your makefile. This is im- 
portant to note if you are using a utility 
other than mwmake to build (such as make 
or clearmake). In this case you'll need to 
invoke mwdip explicitly. 

Again, there are a host of issues that will 
likely cause your initial links to fail. You 
may often run into duplicate symbols, mul- 
tiple included files, unresolved symbols, 
and so on. If you are working with a non- 
MFC project and the linker produces an 
error about the symbol WinMain() being 
multiple defined, you must make sure that 
the makefile TYPE was not incorrectly de- 
fined as MFC, causing the linker to add 
standard MFC WinMain() entry points to 
your code. A similar problem is a multiply- 
defined main() symbol, defined in the file 
MainWin.c, a file that is compiled into all 
MainWin projects. If you have main() de- 
fined in your code, you will need to add 
Example 1(a) to your makefile. Then, the 
file that contains your definition of main() 
includes Example 1(b). This is only nec- 
essary for C++ source files. 


Run-Time Issues 

Run-time problems (aside from your own 
bugs) that you may encounter with a 
newly created MainWin port include: 
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Windows NT language support, memo- 
ry APIs, global handles, byte ordering, 
DDR functions, DCOM, asynchronous 
communication, and various ActiveX API 
functions. Particularly common run-time 
glitches pop up when dealing with the 
registry, programs with multiple threads, 
and customizing the look of your appli- 
cation. 

All MAINWIN registry changes made at 
run time affect only the memory copy of 
the registry file. You can use the API func- 
tions RegCreate. ../RegSet... to make changes 
to the registry at run time. This precaution 
prevents ambiguities due to multiple write 
operations by different MainWin processes. 
To save registry updates between sessions 
set MWAUTO_REG_SAVE to true. 

MainWin assumes that you are work- 
ing in a threadsafe environment. You 
may encounter problems if you run on 
a nonthreadsafe platform. These prob- 
lems are likely to pop up in relation to 
the display (display lockups are com- 
mon). But because developers sometimes 
have to use nonthreadsafe libraries like 
Xlib Release 5 on Irix, and because many 
programs have only one thread, Main- 
Win does not require a threadsafe X li- 
brary. So an application that is not thread 
dependent and runs on nonthreadsafe 
software should work if ported with 
MainWin. 

Finally, it is possible that you won’t 
be satisfied with the default Motif colors 
or the default shadow thicknesses/mar- 
gin widths used by MainWin in the Mo- 
tif look. You can modify the motif look 
of your application by adding the fol- 
lowing X Motif resources in Table 2. The 
values are sample values suggested by 
MainSoft. 


Conclusion 

Reading this might cause you to won- 
der whether it would be better to just 
rewrite your program in Java. I’ll admit 






Table 2: X Motif resources for TestBench. 
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Example 1: If you have main© defined, add this code to your makefile; (b) the 
file that then contains your definition of main) includes this. 


that fighting with compiler issues, make- 
files, and a host of other unforeseen road 
blocks is not my favorite software de- 
velopment activity. In many situations 
Java is a better solution. But in our case, 
we had a significant investment in our 
C++ code base. We weren’t willing to 
give up the many person-years we had 
invested in programming and careful op- 
timization that would be lost in a Java 
rewrite. 

In short, we found MainWin to be an 
adequate solution to the vexing problem 


of porting a large nontrivial NT software 
product to UNIX. 





DDJ 





Listing One 


ifeq ($(COMPILE_OPTION), debug) 
CFGSUFFIX=d 

else 

CFGSUFFIX= 
COMPILE_OPTION=optimized 

endif 


RUN. dir=/TestBench/bin 

PROG = TestBench$ {CFGSUFFIX} 
TYPELIB_FILES = ${PROG}.t1b 
MIDL_TLB=$ {PROG} .t1b 
WRESOURCE = S{PROG}.rc 
TRESOURCE = S${PROG}.rxt 
MIDL_IDL=$ {PROG} . od1 


# Windows resource 
# program 


APP_CCPPFLAGS=-I/testbench/include -IS{MWHOME}/../mfc4@@/include -DRW_NO_STL 


-D_REENTRANT 


SOURCES = \ 
TestBench.cpp \ 
main.cpp \ 


mwupdatedependencies=depend-sunos5 
CPP_OBJS = ${SOURCES:%.C=%.o} 
CPP_OBJS := ${CPP_OBJS:%.cpp=%.0} 


CPP_OBJS := S{CPP_OBJS:%.cxx=%.o} 
OBJS = ${CPP_OBJS:%.c=%.o0} 


Dr. Dobb’s Journal, April 1999 


SRCS = ${SOURCES} 
MAKE VERBOSE=true 
MWUSE_OLE_LIBS = true 
LDEXTERNAL= \ 


-1SimDef${CFGSUFFIX} \ 

-lceddbutil${CFGSUFFIX} \ 

-ltypes${CFGSUFFIX} \ 

-1AltaString$ (CFGSUFFIX} 
\ 


APP_LDFLAGS = 


-lActiveX1${CFGSUFFIX} \ 


-1DLL1$ {CFGSUFFIX} 


__cplusplus = true 


## MFC library related definitions. 


## Removing these definitions disables linking with MFC. 


## MFC_MODULE_TYPE can be one of the following: DLL, EXTENSION_DLL, EXE 


MFC_MODULE_TYPE=EXE 
MFC_LIB_ROOT=mfc 4d 


include $(MWHOME) /make.rules.simple 


DDJ 
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Blarney 
and Steve Jobs 


Michael Swaine 


ince this issue should be getting into 
your hands somewhere between St. 
Patrick’s Day and April Fool’s Day, and 
since I am half-Irish and half-fool, the 
following items, some full of blarney and 
some only half-full, seem appropriate. 

Topics to be covered in today’s lecture 
include: Y2K insight from the I Ching, the 
Blarney algorithm, a trip into Steve Jobs’ 
Reality Distortion Field, and rumors from 
beyond Transmetareality. 

Let’s dispose of the pesky Y2K crisis 
right away. Lest you begin to doubt that 
I am ever-vigilant for fresh perspectives 
on Y2Kgate, I pass along for your edifi- 
cation the following observation, which 
has been taking up space in my e-mail 
inbox since last March: 


The Chinese undoubtedly have the longest 
experience with the Millennium bug. The 
I Ching was worked out 5000 years ago, it 
has a 6-bit word and the carry bit was nev- 
er invented. 

—Pushkar Piggott 


Mr. (Ms.?) Piggott doesn’t report what 
the I Ching has to say about the Millen- 
nium bug, but the fact that it has contin- 
ued to operate through 50 century-counter 
rollovers is encouraging. 

Perhaps the I Ching can help with a 
particular Y2K problem that I just read 
about. 

Apparently you can save yourself mon- 
ey by buying your tombstone before you 
actually need it, on the assumption that 
stone will appreciate in value faster than 
your programming skills. It's also just more 
convenient to make purchases while 
you're still alive. 

And apparently you can save even more 
if you get as much of the carving as pos- 
sible done ahead of time. This requires 
that you make certain assumptions, such 
as that you will remain the beloved father 
of etc. You can see where this is leading. 


Michael is editor-at-large for DD]. He can 
be contacted at mswaine@swaine.com. 


Dr. Dobb’s Journal, April 1999 


ROGRAMMING PARADIGMS 


A lot of people took this stone-carving 
cost-cutting advice to such an extreme 
that they had “19” carved into stone as the 
first two years of their death date. Now, 
unless they really luck out and die in the 
next eight months, they face having to 
change the carving. And it’s really hard to 
erase in this medium. Kudos to the pro- 
grammer who first comes up with a soft- 
ware patch for this one. 


Only Half Blarney 
It's a great story. Even if it isn’t entirely true. 

Only once in a blue moon do the dead- 
tree news sources give front-page treat- 
ment to the development of a new algo- 
rithm. It’s not as easy a subject to cover 
in journalese as, say, candy-colored com- 
puters. So it’s particularly entertaining that 
when the Times of London and various 
other newspapers decided to play up a 
new algorithm, it was an encryption al- 
gorithm. 

Here’s what I mean: With enough time 
and patience, you could probably explain 
a sort routine to any reasonably bright 
person. You could also probably get 
across the gist of a graphics compression 
algorithm to most of a class of tenth- 
graders, if they'd sit still for it. But en- 
cryption is something else; if you start try- 
ing to explain what’s going on in a 
public-key encryption algorithm, you 
could easily find yourself bogged down 
in linear algebra, number theory, combi- 
natorial mathematics, Fourier analysis, cod- 
ing theory, complexity theory, statistics, 
or probability theory. I suppose that in- 
spired explainers can get the basic con- 
cepts across to a broad audience, but it 
ain’t easy. Encryption is obscure. Which 
is perhaps why they got the story wrong. 

“An Irish teenager has developed a fast 
security-encryption mechanism for e-mail 
that has the potential to replace the 22- 
year-old system in use today,” says Tech- 
Web News. 

“Making your e-mail secret is now 30 
times faster, but the innovation has come 





not from a multinational computer but 
a schoolgirl from Blarney, Ireland,” said 
the BBC. 

“A 16-year-old Irish schoolgirl is being 
hailed as a genius for developing a way 
of encoding data on the Internet 10 times 
faster than systems used globally now,” 
said Reuters. 

Most of what they report is true. Sarah 
Flannery is 16, or was at the time of the 
stories, and she is Irish, from Blarney, in 
County Cork. She is also very bright. She 
has won more than one award in presti- 
gious international science fairs, one of 
them landing her a trip to Texas and a 
handshake from semiconductor legend 
Gordon Moore last year. 

And she did come up with a public-key 
encryption algorithm that is faster than 
RSA. But it appears that the newspapers 
(but not Flannery) may have exaggerated 
a little about how original the work is and 
how useful it is. 

Flannery began the work on the algo- 
rithm while working at Baltimore Tech- 
nologies in Dublin on a student work 
placement in March, 1998. (Baltimore, a 
major name in encryption, merged in Jan- 
uary 1999 with the British encryption com- 
pany Zergo.) According to William Whyte, 
Senior Cryptographer at Baltimore, the 
company gave Flannery the problem to 
work on. “We’ve been looking at algo- 
rithms based on 2x2 matrices for a while,” 
he said “and gave her the idea to see what 
she could do with it.” What she did, ap- 
parently, was to work out a public-key al- 
gorithm that is considerably faster than 
RSA, and equally secure. 

This is impressive, and she no doubt 
deserves all the scholarship and job offers 
she’s been getting. And I applaud her de- 
cision not to try to patent her algorithm. 
But it may be no more than cryptogra- 
phers at Baltimore had already achieved, 
and it may have drawbacks that make it 
impractical as an alternative to RSA. As 
the Baltimore cryptographers worked it 
out, “both the key and the ciphertext are 
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about eight times the length of the 
modulus, rather than more-or-less the 
length of the modulus as with RSA,” 
Whyte said. 

Her father, a mathematics instructor at 
the Cork Institute of Technology, has tried 
to place the achievement in some per- 
spective. “Sarah has a very good under- 
standing of the mathematical principles 
involved,” he said, “but to call her a ge- 
nius or a prodigy is overstated...” 


(Hence the “Western. | the “Standards” 
part owed to the fact that SWAC was — 
built for the National Bags of Stan- = 


fine place 2 to build a compare: sain NBS 
had a lab at UCLA. Well, they had < 

empty room, anyway. SWAC, whose | 

power supply alone took up a whole 


ful work for NBS, ahd saieae the 
material for a lot of reminiscences by 
those who worked on it. One of the 


members fondly tl 
he took in the warm, dea be- 
hind the bays of the huge SWAC. 
Try doing that we oe Gateway PC. 
M.S. 
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The algorithm is called the Cayley- 
Purser algorithm, after Arthur Cayley, a 
19th century British expert on matrices, 
and Michael Purser, a cryptographer from 
Trinity College, Dublin. Since the algo- 
rithm is based on matrices, the tribute to 
Cayley is understandable. He developed 
the algebra of matrices, published the first 
English contribution to the theory of de- 
terminants, and invented the notation of 
two vertical lines on either side of the ar- 
ray to denote the determinant. Purser is a 
modern mathematician, whose _ best- 
known work may be his book Secure 
Data Networking (Artech House, 1993; 
ISBN 0-890-06692-2). 

The algorithm uses 2x2 matrices, where 
each entry is an RSA number. Because of 
this, the level of security it offers is ex- 
actly the same as that of an RSA key with 
the same modulus. But the algorithm sub- 
stitutes a small number of matrix multi- 
plications for the modular exponentiation 
required by RSA, which is where it gets 
its speed. The trade-off, though, is a con- 
siderably longer key and ciphertext. At 
least that’s what the Baltimore cryptogra- 
phers found. Maybe Flannery has im- 
proved on their work, but we won’t know 
until she publishes. 


Five Fruit Flavors... 

I went to MacWorld Expo in January and 
subjected myself to the emanations from 
the best salesman in the industry. Here’s 
what I found in my notebook when I 
came out from under the influence: 

Wasn't that the Woz who cut across in 
front of me a few minutes ago while I was 
waiting in the Press/VIP line? Anyway, 
that’s Dan Gillmor of the San Jose Mer- 
cury News two seats down from me, so 
I’ve got the same perspective as the Merc 
on this dog-and-pony show. Gillmor says 
something about the keynote not starting 
on time and I make some lame joke about 
all our watches automagically resetting 
themselves when Steve Jobs walks on 
stage. His legendary Reality Distortion 
Field, you know... 

The show starts with a commercial, as 
Apple conferences often do. Hal 9000 
blaming Y2K on people who didn’t buy 
Macintosh. Jobs comes onstage to the 
same music that introduced the monolith 
in 2001. He lays out the specs for the new 
Yosemite line of G3 Macs. Up to 400-MHz 
processors; the first copper semiconduc- 
tor technology in a commercial comput- 
er; built-in ATI Rage 128 graphics accel- 
erator; 16-MB graphics memory; OpenGL 
library; up to a Gig of RAM; up to three 
hard-disk drives inside the case for 100 
GB of internal storage; four slots of which 
three are 64-bit PCI, 100 megabit Ether- 
net with 1-GB optional; USB and 
Firewire... 


Then he shows the machine. It’s beau- 
tiful. iMac colors, but a more sophisticat- 
ed, professional look. A door that makes 
access remarkably easy. Three really at- 
tractive monitors. Systems starting at $1599 
and available today. The colors and de- 
sign are, I suspect, way more important 
than most people in the industry realize... 

The big system software news is the gen- 
eral release of MacOS X Server. This is the 
server version of OS X, the client release 
being slated for later this year. It's what’s 
left of Rhapsody, and it’s a snapshot of 
where Apple’s at in terms of slipping 
NeXT’s UNIX kernel and other technolo- 
gy in under the Mac GUI to bring the Ma- 
cOS up to date. It looks like MacOS and 
seems to do things you wouldn't expect 
from MacOS. Included: the NeXT Mach 
kernel, Apache, WebObjects, BSD 4.4, Java. 
$995 server license, shipping in February 
(didn’t quite make the Expo ship target). 
Bundling WebObjects is a big concession 
for Jobs, since this web app development 
software sells well now for thousands of 
dollars a pop. But is it the full WebObject? 
Skeptical minds want to know... 

From the darkness at the back of the 
stage a huge rack rolls out with seven rows 
of seven iMacs. The 50th is on the podi- 
um in front of Jobs, and all 50 are run- 
ning software disklessly from a single new 
Yosemite machine running MacOS X Serv- 
er. Jobs fires up a different full-screen 
video on each machine, and it is very im- 
pressive. Perhaps the most interesting 
point in the demo, though, is not the pow- 
er of MacOS X Server, but the fact that all 
iMacs run a tweaked version of the Mac- 
OS that allows them to boot from the net- 
work. Every iMac today has a hard disk, 
but all the technology for a diskless iMac 
is apparently in place. Thin clients at a 
lower price are only a decision away. The 
thumbprint of Apple board member Lar- 
ry Ellison is evident here... 

Okay, the color iMacs. The iMac, Jobs 
reminds the crowd, is the best-selling com- 
puter in America. 800,000 sold between 
August 15 and December 31. According 
to Apple’s December sales stats, about a 
third of those buying iMacs are first-time 
computer buyers, “the most coveted cus- 
tomers in the industry.” About three quar- 
ters of the iMac buyers are not replacing 
another Mac. That means that Apple: 1. 
has made very credible inroads into the 
consumer market; and, 2. has pushed be- 
yond its installed base... 

The colors, the colors! Right. Five col- 
ors, all cool, all named for fruits. Avail- 
able right now. Provocative new TV ads. 
Other iMac news or the lack of same pales 
in comparison. A price drop to $1199, 50 
percent more disk storage, a faster pro- 
cessor. Yadda yadda...No, it’s the color. Or 
the colors. 
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What’s the big deal about color com- 
puters? Jobs gets fairly silly raving about 
the colors. But I really think the iMac and 
Yosemite designs and colors mark some- 
thing new for the computer industry. A 
fundamental shift in the nature of the busi- 
ness. I wish I knew just what it was; I’d 
invest, but I have a theory... 

The rest of the keynote was about third- 
party software. Or what Apple is doing to 
support same. Getting John Carmack, 
founder of Id Software and God of Games, 
onstage to say that the Mac now had ev- 
erything a game designer or player needed 
wasn't a bad move. Ditto licensing OpenGL, 
a move urged by game developers. And 
Connectix has a Sony Playstation emulator 
for the Mac. I sense a theme here... 

The big hardware news, in terms of 
third-party opportunities, is that Apple is 
embracing standards and changing pro- 
tocols. PCI, USB, Ethernet, and Firewire 
device makers will benefit... 


Color Reflections 

After the Expo, I saw a few cracks in the 
image. Firewire, hot technology though it 
may be, ran into immediate trouble when 
it became clear that Apple wanted to charge 
a pretty hefty licensing fee for it. If Apple 
doesn’t back off, the technology may not 
take off as quickly as Apple hopes — or 
needs it to. 

And I confess embarrassment at treating 
color as more significant than technology, 
but I don’t recant. That is, I think that the 
color and design decisions that Apple has 
made are potentially more significant for Ap- 
ple and perhaps for the industry than any 
technical advance Apple could have made. 

Turning color and design into significant 
considerations in buying a computer takes 
the industry a step further in the direction 
of commoditization of the box. Commodi- 
tization is usually understood to mean in- 
terchangeable gray boxes distinguished only 
by price. But that’s not how people buy 
commodities; they do care about design 
and color, even— perhaps especially— in 
big-ticket items, like cars. Design matters 
in commodities. Color matters. 

Jobs is taking this fact seriously. No one 
has really done this before. There have 
been color computers before, and there 
will be attempts to imitate the Apple col- 
orization, but no one has set out as de- 
liberately to transform the public percep- 
tion of what a computer is since—well, 
since the last two times Jobs did it. 

This oughta be interesting. 


Transmeta Blarney 

There was some amusing disinformation 
on the Web not long ago suggesting that 
Silicon Valley’s most mysterious startup 
was going to apply its hot new technolo- 
gy to—which platform? Windows 2000? 
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Linux? MacOS X? No, to the Amiga. At the 
risk of offending the Amiga community 
again, it was a pretty entertaining story, 
but apparently entirely untrue. No disin- 
formation here, I hope, just a frank ad- 
mission of a lack of information. Like most 
everybody in Silicon Valley, I don’t know 
what Transmeta is up to. 

Several facts are known about the hush- 
hush startup. Its address: 3940 Freedom 
Circle, Santa Clara. Its telephone number: 
408-327-9830. Its tantalizingly mute web 
site: http://www.transmeta.com/. And the 
fact that it has hired Linus Torvalds, cre- 
ator of the Linux operating system. 

Okay, so maybe it’s not so mysterious. 
The story is that they make VLSI chips for 
the multimedia market. Really hot chips. 
IBM Microelectronics will be building chips 
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for them in IBM’s state-of-the-art silicon fa- 
cilities. Fine. But why, then, is Transmeta 
hiring engineers skilled in kernel develop- 
ment and compilers? And what’s Torvalds 
doing there, anyway? And Bill Rodriguez 
of the MIT AI Lab’s Project MAC? 

One person who probably knows all 
the answers is Microsoft cofounder and 
high-tech angel Paul Allen, who has in- 
vested heavily in the company. But I have 
a hunch he’s not talking. 


Next Month 
Come May, I'll be on special assignment 
and, for the first time in a decade, this col- 
umn will be absent. I will, however, re- 
turn in June. 


DDJ 








To Securing 


Software. 


Peer Aichi 
Chatiesth trtertded 


www.rainbow.com/g10 is where to get it. Free. 


Piracy is at an all-time high. And if you’re not using software security products 

from Rainbow, you're in danger of becoming a statistic. We’re the world’s 

number-one source for software protection, and we’ll give you a free segs: 
ra 


that illustrates why. Get yours by visiting our web site today. It’ll h 


you're around tomorrow. 


e Software activation via the Internet. 

e Sentinel software protection is 
portable, easy to install and reliable. 

e A single Sentinel key securely pro- 
tects up to 14 different applications 
with unique encryption algorithms. 





e Parallel port, USB, ADB and RS/232 


GFRAINBO 


TECHNOLOGIES 


p ensure 


e Over 15 million Sentinel keys worldwide— 


more than 50% of all protected software 
has a Sentinel key. 


compatible. 












Call for 


USB design Changing the way the world secures business. your low cost 
Developer's Kit. 


50 Technology Drive, Irvine, CA 92618 
800.705.5552 tel. 949.450.7300 fax. 949.450.7450 www.rainbow.com 
Offices and distributors located worldwide. 


* International Planning Research, 1998 


93 











The thing that makes Software Development ‘99 


so exciting is the very same thing that makes it 


different from any other conference. 


SD ‘99 is the only major forum for development 
professionals that is truly independent. Instead of a 
single point of view, you'll hear industry leaders 
representing a variety of technologies and a 
spectrum of competing visions. Then you'll 
evaluate and extract the truth for yourself. 

SD ‘99 isn’t a marketing event. It’s about keeping 
developers ahead of the curve, well-rounded and 
more effective. Which is why, unlike any other, this 
conference has flourished for over a decade. 

This year, choose from over 200 classes and tuto- 
rials and test-drive the latest tools from over 250 
vendors. Join thousands of developers and their 
managers at the most critical event in the industry — 


the one that focuses on developing the developer. 


CONFERENCE TRACKS 

- C++ Programming 

- Database & Middleware Development 
- D/COM(+) Programming 

- Java Programming 

- Internet Development 

- Management Practices 

- Methods & Modeling 

- Object & Component Development 

- User Interface Design 


- Windows2000 Platform Development 





Receive a limited edition 
SD'99 fleece jacket 
with every VIP Registration 


Miller Freeman 





VISIONARY KEYNOTES 


Sponsored by 


Harmony: Business, Technology & Life 
After Paperwork 

Arno Penzias, Former Vice President & 
Chief Scientist, AT&T Bell Laboratories 


Modeling Web-Based Systems Using UML 
Grady Booch, Chief Scientist, Rational 
Software 


(Sas). s gti i ilesiiais waewuieis = 8) alias ae) 
ya¥s)s)i(e-19(e)4) wii sie) ataee 
Steve Lucco, Senior Researcher, Microsoft 


C++: Are We Having Fun Yet? 
Andrew Koenig, AT&T Research 











TECHNICAL BRIEFING 
KEYNOTES 


% SUN 


TUCTOSYStEMs 


Java's Advances in 1999 

Jim Mitchell, Vice President of 
Technology & Architecture for 
JavaSoft & Sun Fellow 


iBi: 


Building E-Business Applications 
John Swainson, General Manager 
of IBM Application Enabling & 
Integration 


Micresoft 


Adam Bosworth, General Manager 
of Weblications, Microsoft 




















SPECIAL EVENTS 


Java Super Bowl Vil 





THE EXHIBITION 


Special Pavilions 


__ Meet more than 250 leading 
tools vendors including: 


Technical Sessions 








DEVELOPMENT 


hYclil maclaaiae. 
\/Kohyerole (=) @ =i ends 7g 
me callojnace a 
May 10-12 


Conference 
May 9-13 











C PROGRAMMING 


Java Jive and 


Scrolling the Editor 


Al Stevens 


hat do you do when you live in 
a “dead spot”? That’s what AT&T 
told me. I live in a dead spot. It 
all started when I learned that for 
a flat monthly fee I could get a cell phone 
that includes some number of minutes of 
long-distance calls from anywhere to any- 
where in the continental U.S. and no 
roaming charges. My current service does 
not include the long-distance feature and I 
pay roaming charges whenever I use the 
phone outside the reaches of BellSouth Mo- 
bility. Some simple arithmetic revealed that 
my average monthly long-distance bill and 
my local cell-phone charge would be more 
than offset by the AT&T all-encompassing 
monthly charge. The roaming charge re- 
moval was an added incentive because I 
travel a lot. 

I called the local AT&T rep and signed 
up. He sent me a pocket-sized cell phone 
with instructions on how to set it up. It 
seemed to work except that my voice and 
the voice of whomever I called broke up 
a lot, so much so that meaningful con- 
versation was not possible. 





“Can I spea...to Char...? 
"Wha..:2 Ica...hear:.. You're break:..: .<:p.” 

I called the AT&T sales rep who told 
me to call tech support (on my land line, 
of course). The nice lady who took my 
call began by asking how many stair steps 
do I have? This is a one-story house, I an- 
swered, no stairs. No, no, she said, the 
cell phone has a little LCD stair step dis- 
play to indicate the signal strength. The 
picture of the phone on the box it came 
in had five stair steps. I didn’t have any 
stair steps on my phone. We discovered 
that if 1 went to different parts of the house 
I could get one or two stair steps inter- 
mittently. Not enough consistent stair steps 
to complete a call. Do I have a tin roof? 
No, but I have a cat. She sent me out into 


Al is a DDJ contributing editor. He can be 
contacted at astevens@ddj.com. 
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my yard (my land line has a portable 
handset, so I took her outside with me). 
From my driveway I could get two stair 
steps more frequently than not. Would I 
be willing to make all my calls from the 
driveway? No, we get a lot of rain. 
Finally, we agreed that I just don’t get a 
strong enough signal at my house. She 
asked where I live exactly and then blamed 
the problem on my proximity to Kennedy 
Space Center (15 miles away) and Orlando 
International Airport (40 miles away). All 
those antennae and low-flying aircraft and 
space shuttles, she said. I said that was sil- 
ly. People use their cell phones all over the 
airport and at the Space Center. You see 
them walking around making important 
calls all the time. After sending me a re- 
placement phone to try, which did not work 
either, AT&T decided that I live in a “dead 
spot” and asked for their phone back. 
This is depressing. First, you are told 
you live in a dead spot, and you go into 
denial. This lasts for a while as you see 
others using their cell phones in their cars 
and at the mall, people who obviously do 
not live in dead spots. Eventually, you re- 
alize that you need help. The first step to- 
ward recovery is to admit to yourself that 
you live in a dead spot. After that it’s one 
day at a time. I can’t find any support 
groups or programs for people who live 
in dead spots. If anyone of you has the 
same problem maybe we can get togeth- 
er and help one another. When you feel 
the effects of living in a dead spot getting 
the better of you, you can call me any 
time of the day or night. I'll hurry over to 
your house and stay with you until the 
crisis passes, during which we can get 
drunk together. Speaking of cell phones... 


What the Heck is Java, Anyway? 

About 50 years ago, the Ink Spots sang, “I 
love coffee, I love tea. I love the java jive 
and it loves me.” Java jive. How prophetic. 
I thought Java was an improved program- 
ming language that looks something like 
C++ but with a more object-oriented pro- 
gramming model and fewer of the lega- 
cy C idioms that trip up programmers, a 





language intended to be interpreted and 
platform-independent so that programs writ- 
ten in Java can be run without dependence 
on a specific platform. That's what I thought. 

The “Preface” to The Java Application Pro- 
gramming Interface, Volume 1, (Addison- 
Wesley, 1996) by James Gosling, et al. says, 
“Java is a general-purpose object-oriented 
programming language.” The book goes on 
to explain that Java evolved from a language 
designed to write consumer device em- 
bedded applications into one for writing 
platform-independent applets that could 
be run from within HTML pages by Java- 
enabled web browsers. This three-year-old 
explanation tends to reinforce what I 
thought. 

Recently on 60 Minutes, Scott McNealy, 
CEO of Sun Microsystems, said something 
that challenges that idea. When CBS cor- 
respondent Lesley Stahl asked if Java was 
going to make Windows obsolete, he said, 
“Windows is obsolete,” and went on to 


say that people don’t need Windows any- 


more, that all they need is Java as the en- 
gine that drives their computer-based ap- 
pliances. He illustrated this notion with a 
cute little cell phone that opened like a 
clam shell to expose a tiny screen and 
keyboard similar to a Windows CE HPC 
device. Only it wasn’t Windows CE, it was 
something else. Java, we are led to be- 
lieve, running a Swiss Army personal in- 
formation and communication device. 

In an unrelated article about Microsoft's 
problems with the Department of Justice, 
Charles Cooper of ZDNet Tech News said, 
“\..this was a time when excitement was 
building [at Microsoft] around Java as an 
alternative to Windows.” 

What? What are these guys talking 
about? Java can’t be an alternative to Win- 
dows. Windows is one of the foundation 
platforms that support Java. The alterna- 
tives to Windows are UNIX, OS/2, the Mac 
OS, the embedded OS of your favorite ap- 
pliance, and so on. But these spin doctors 
seem to be telling us that Java is itself a 
full-blown operating platform and not a 
programming language that rides on some 
other language-independent platform. I’ve 
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seen TV ads that convey that same false 
impression, too, as if there is some need 
to persuade the public to get on board 
and embrace Java instead of Windows. 
But then I read Sun Microsystem’s own 
Java Platform web page (http://www.sun 
.com/java/platform.jhtml), which says: 


If you're reading this in a web browser on 
a personal computer or workstation, you’ve 
probably already got the Java platform. It’s 
incorporated into all major web browsers. 


So what am I missing? Give someone a 
computer and Java and what do they have? 
Nothing useful. Java needs more, something 
underneath, doesn’t it? Java all by itself does 
not understand display devices, file systems, 
pointing devices, and so on. In other words, 
is no more or less an operating platform 
than C++ or Basic are operating platforms. 
You have to have an operating system. You 
have to have an operating environment. “All 
major web browsers” implies not only an 
operating system, but device drivers and a 
graphical user interface to integrate the dis- 
play and input devices into a common user 
interface and onto the hardware platform. 
Then you need the web browser software 
itself on top of the operating environment 
and underneath Java. Which says that Java 
is at least two layers removed from Win- 
dows in the hierarchy of what you need to 
run programs on a PC. 

Am I wrong? No, I’m right, according to 
Sun, itself. In “What is the Java Platform?” 
(http://java.sun.com/docs/white/platform/ 
javaplatform.docl.html), Sun says: 


The computer world currently has many 
platforms, among them Microsoft Windows, 
Macintosh, OS/2, UNIX and NetWare. ..what 
sets the Java Platform apart is that it sits on 
top of these other platforms... [italics added] 


There is nothing particularly new and 
innovative about a language translator that 
sits on top of an operating system and in- 
terprets bytecodes (remember Pascal? 
SmallTalk?), but the point is that Sun’s own 
online documentation contradicts what Mc- 
Nealy spoon-fed to 60 Minutes viewers. 

Agreed, you don’t need Windows specif- 
ically to run Java applets, but, on a PC, 
what are the alternatives? Linux and OS/2 
come to mind, but that’s not McNealy’s 
point when he says Windows is obsolete. 
He implied that his littke multifunction cell 
phone and all the other Java-enabled con- 
sumer appliances—TVs, pagers, electric 
toothbrushes, vibrators, whatever— have 
eliminated the need for desktop PCs, which 
is why, in his view, Windows is obsolete. 
Windows is obsolete, he says, because you 
don’t need the machine that Windows runs 
on. Maybe someone should make McNealy 
do a spreadsheet on that tiny keyboard; or 
write a report; or view anything of sub- 
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stance on the web. Maybe someone should 
explain to him the difference between his 
own Java Base Platform and the still- 
evolving Embedded Java Platform, which 
are described in the same web page. 

What I saw on 60 Minutes was typical 
flim-flam: McNealy confessing his techni- 
cally challenged background; McNealy giv- 
ing a speech while cruising the stage on 
rollerblades; McNealy playing hockey to 
show what a hard-driving, competitive 
guy he is; McNealy talking down-home 
trash and taking advantage of the relative 
technical naiveté of the viewing public to 
promote his company’s agenda, which 
these days is to bash Microsoft. Lesley 
Stahl bought it hook, line, and sinker, so 
maybe others will, too. Not us program- 
mers, I'll wager. 

I do not mean to belittle the importance 
of Java as a development platform that spans 
operating platforms and runs applications 
identically everywhere with the Internet as 
the ultimate operating platform. I hope and 
believe that Java, or something that falls out 
of it, will achieve that objective. But Java as 
an alternative to Windows? That’s like buy- 
ing a steering wheel instead of a car. 


Editor Fixes 
In January, I introduced the Editor project, 
a programmer’s editor that I will eventual- 
ly integrate into Quincy 99, the Windows- 
hosted IDE that serves as a front end to a 
GNU C/C++ compiler, a project that I 
launched in December (http://www. mid- 
ifitz.com/alstevens/quincy99/). I built a cus- 
tom editor class to support large text files, 
which CEdit does not do, to provide a com- 
prehensive undo/redo feature, which CEd- 
it and CRichkdit do not have, and to have 
an editor that is independent of the Win32 
API in case I decide to port Quincy 99 to 
a different platform some day. Having strug- 
gled with text editors in the past and be- 
ing aware of their particular problems, I 
wanted to use the sfd:: container classes to 
implement text documents in memory. It 
seemed that such an approach would make 
the problems of accounting for lines of text 
simpler to manage. I found that to be true. 
By now, several readers have down- 
loaded the editor (http://www.midifitz 
.com/alstevens/editor/) and experiment- 
ed with it. Some sent comments and bug 
reports, which I addressed in code and 
uploaded. The most prominent problem 
was caused by my requirement to support 
longer files than CEdit could manage. 
Whereas I addressed the CEdit limit of file 
size based on byte count, I failed to ad- 
dress a problem in the Win32 API that 
shows up with files that have a lot of lines 
of text. Until I heard from the readers, I 
didn’t even know that the bug was there. 
The solution is simple but was also un- 
known to me. 


The problem is with scrollbars and scroll 
thumbnail buttons. A program typically 
uses the CWnd-.:SetScrollRange function 
to set the range of values that represents 
the length (and width, too; I'll just discuss 
the vertical scrollbar here) of a document 
as it relates to the scrollbar. Text docu- 
ments represent their lengths in lines of 
text. The program does this when it loads 
the document and whenever the number 
of lines of text changes because of edit- 
ing actions by the user. The program then 
uses the CWnd::SetScrollPos to set the po- 
sition of the thumbnail button to be some 
value within the range, usually the line 
number of the line of text that is displayed 
at the top of the window. The program 
does this whenever the user scrolls or 
pages the document. The problem is that 
those functions use int parameters to rep- 
resent the range and position, and ints 
under the Win16 compiler were 16 bits. 
Since the run-time part of the API has to 
support Winl6 programs, the arguments 
are kept within the range of a 16-bit in- 
teger even though an int under Win32 is 
32 bits. The CWnd::OnVScroll and 
CWnd::OnHsScroll message notification 
functions similarly report the scroll posi- 
tion as a 16-bit quantity. 

The solution, for which I am indebted 
to readers Ben Constable and Gerald 
Schwartz, is to use CWnd.:GetScrollInfo 
and CWnd::SetScrollfo. The application’s 
overridden OnVScroll and OnHScroll func- 
tions ignore the UJNT nPos argument that 
Win32 passes and call CWnd::GetScrollInfo 
to get the nTrackPos data member into a 
SCROLLINFO structure to use as the scroll- 
bar’s position. To set the thumbnail posi- 
tion, the program initializes the nMin and 
nMax data members of a SCROLLINFO 
structure and passes the structure’s ad- 
dress to the CWnd::SetScrollinfo rather 
than calling CWnd::SetScrollRange. 

Ben and M. Werner both reported a prob- 
lem with how I was dealing with the size 
of the text font, a problem related to plat- 
forms other than what I have here. They 
both offered code to correct the problem. I 
really like this kind of beta testing; the testers 
not only find the problems, but they fix them 
too. Doesn’t leave much for me to do. 

Thanks also to reader Jean-Francois Pail- 
lard for reporting and fixing a really stupid 
bug. When I implemented the text search 
feature I included the ability to search for- 
ward and backward like most text editors 
do. The only problem was that I didn’t 
bother to test the backward operation 
when searching for a string that does not 
exist in the text. Naturally, that was the op- 
eration that killed the program. Well, what 
else could you expect from a programmer 
who lives and works in a dead spot? 


DDJ 
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Are Java Applets 
Independent Programs? 


Steve Ball and John Miller Crawtord 


Java applet may seem to be an in- 
dependent program, but in one cru- 
cial aspect, it is not. If you change 
a static field of a class used within 
an applet, that change pervades all 
applets — even if they are on different web 
pages. Although, as we showed in “ Chan- 
nels for Inter-Applet Communication” 
(DDJ, September 1998), this behavior 
opens up a channel for inter-applet com- 
munication, such effects can be as un- 
welcome as they are unexpected. 

An example CAD system we wrote, 
called “WebCAD,” which lets models be 
built and displayed in 3D, demonstrates 
these unwelcome effects. (The complete 
source code and related files for WebCAD 
are available electronically; see “Resource 
Center,” page 5.) First, consider this sys- 
tem implemented as a stand-alone Java 
application. Figure 1 shows two instances 
of this application, in separate windows. 

At its lowest level, WebCAD deals en- 
tirely in polygons (or planes). The mod- 
el is displayed by drawing each plane in 
turn. Users can zoom in or out of the pic- 
ture by changing the scale factor, set the 
display to either wireframe or solid- object 
mode, and spin the object around its x- 
and y-axes by dragging it with the mouse. 
All of these settings necessarily affect ev- 
ery plane in the model. 

In WebCAD, we use static fields in the 
Plane class to express these display 
properties — one for the scale factor, an- 


Steve is an independent contractor special- 
izing in C++/Java/UNIX development. He also 
lectures in Object-Oriented Design and C++ 
programming at the Auckland Institute of 
Technology, New Zealand. Steve can be con- 
tacted at steve@effectivejava.com. John is In- 
teractive Technology manager at Dave Clark 
Design Associates and principal lecturer at 
the Auckland Institute of Technology. He can 
be contacted at john.m.c@ait.ac.nz. 
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other for the display mode, and two to- 
gether for the current viewing angle; see 
Listing One. This is the alternative to stor- 
ing the display properties as nonstatic 
members in each Plane object, which 
would require iterating through all Plane 
objects and changing each individually — 
a task that is both tedious and needless- 
ly time consuming, since it would be 
meaningless for any two planes to be dis- 
played with a different scale factor or 
viewing angle. 

Used this way, static fields are fulfilling 
the role for which they were originally 
created in C++ (from which Java inherit- 
ed them). According to Bjarne Stroustrup 
and Margaret Ellis in The Annotated C++ 
Reference Manual (Addison-Wesley, 1990), 
“A static member function or variable acts 
as a global for members of its class with- 
out affecting the rest of the program.” 
They go on to say that since “a static data 


-member [is] shared’ by all objects of a class 


in a program,” their scope is limited to the 
executing program. 

The consequence for WebCAD is that 
if we have two windows, each display- 
ing separate instances of it (see Figure 1), 
we can change their display properties 
independently even while the planes in 
each separate model share the same 
values. 

In contrast, if we make a direct trans- 
lation of WebCAD to an applet and dis- 
play separate instances of it on separate 
web pages, we cannot make independent 
changes to their display properties. The 
browser executes applets as separate 
threads in the same Java Virtual Machine, 
so all applets share the same values of the 
static fields of any common classes. The 
scope of the static fields is not limited to 
the applet object or its executing thread, 
but encompasses all objects of the applet’s 
class, wherever they are instantiated by 
the browser. 





Figure 2 shows two applets experienc- 
ing just this problem. The models should 
be in different scales and different display 
modes, but they both have identical ap- 
pearances. 


Reclaiming Static Semantics 
for Applets 
What you need for applets, then, is some 
means of resolving the sibling rivalry that 
occurs when applets wish to have their 
own private copies of static fields. In this 
article, we present a workaround that lets 
the conventional semantics of static fields 
be reclaimed through the use of a spe- 
cialized mechanism that mimics the be- 
havior of static fields. More importantly, 
this workaround possesses the same mean- 
ing in both Java applications and applets. 
This lets you develop classes that op- 
erate identically, whether instantiated with- 
in an application or an applet— something 
not possible if the classes use static fields. 
First, we present our solution in code form, 
and then we abstract its essence in the form 
of a design pattern. We'll also provide a 
set of rules that may be mechanically ap- 
plied to a class that uses static fields, which 
will convert it to a class that is portable be- 
tween executing environments. 


Mimicking Static Fields 

Because static fields may not be used to 
store values particular to just one applet, 
it follows that these values need to be rep- 
resented as instance variables in some ob- 
ject. The main Applet object is an excel- 
lent candidate for the custodian of these 
values, both because it sits at the root of 
the function call hierarchy and because 
there is only one per applet. 

However, the corollary to this decision 
is that in some form these values have 
to appear in every stack frame from the 
topmost to the bottommost. We will dis- 
cuss three ways to do this. 
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Figure 1: Independent objects in separate application windows. 
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First, they may be passed as construc- 
tor arguments to each class (see Listing 
Two) in the composition hierarchy. 


Eliminating Argument Clutter 

The second solution seeks to avoid the 
passing of long lists of values to a partic- 
ular class (such as Cube) with no other 
purpose than to allow that class to pass 
them on down. Instead, you can simply 
pass a reference to the applet itself; see 
Listing Three. 

This approach of placing the values 
in the Applet object is adequate for many 
situations where objects depend upon 
global state values particular to just their 
applet. Consider, for example, an applet 
that supports an HTML parameter that 
allows sound effects to be turned on or 
off. The setting of this value needs to be 
available to a variety of methods in a 
number of different low-level classes. If 
the value is stored in the applet during. 
its initialization, then all objects with ac- 
cess to the Applet object are able to 
quety it. 

We recommend that you routinely pass 
the main Applet object as a constructor ar- 
gument to all helper classes. Then, when 
you discover you need to configure the 
applet’s behavior at some low level based 
on a global value, you do not have the 
arduous task of retracing your steps back 
up the call hierarchy and inserting Applet 
references and constructor arguments into 
every intervening class. 


Emphasizing Relationships 
Between Classes 
The third approach is suitable for imple 
cases, particularly where global data (ike 
sound-effects flags) are not clearly owned 
by any one class (other than the applet). 
The disadvantage is that the Applet ob- 
ject, like the constructor argument list in 
our first solution, becomes cluttered with 
the shared values from all the other class- 
es in the application that require applet- 
specific values to be stored. Also, there is 
no explicit relationship between the mys- 
terious appearance of an instance variable 
in the Applet object and the class of the 
objects that depend on it. 





Figure 2: Interference between objects 
in the same browser window. 
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that affect them all. That structural rela- 
tionship may be expressed by encapsu- 
lating the display properties in a separate 
class, an instance of which is placed in 
the main Applet object and passed (even- 
tually) as a constructor argument to each 
Plane object; see Listing Four. 
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and the class that holds its static fields 
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Independent Applets in Java 
WebCAD is implemented as an applet that 
may be safely instantiated multiple times 
in the same browser window without any 
unpleasant inter-applet interference. 
Figure 3 is a web page with two in- 
stances of the applet displayed. The in- 
stances have entirely different display 
properties: One has a wireframe appear- 
ance, the other is solid with hidden-line 
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Figure 3: Independent objects in the 
same browser window. 
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Motivation. It is primarily in the con- 
text of Java applets that the Shared Prop- 
erty pattern is most usefully implement- 


Intent. Allow the sharing of properties 
within a defined subset of the set of all 
objects instantiated from a class. 


by other applets, and may be generalized 
in the form of the Shared Property design 
pattern. 




























































































#include "iostream.h" 


void f( int i ) 
{ 


if( i !t=4 {| i !e= 


== 


ttern. 


for C/C++ 


Version 7.5 
presents Bug # 774 


5 ) return; 


cout << "hello world\n" 


} 


int main() 
{ 
f{ 32 );3 
£( 4 }>; 
£{ -5 3% 
return 0; 


} 


The programmer was surprised to receive no output. What's going on? Call if you 
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ed, since web browsers’ implementations 
allow Java class variables to be shared 
across applet boundaries. The Shared 
Property pattern mimics the behavior of 
static fields in classes executed within in- 
dependent Java applications. 

Applicability. Use the Shared Proper- 
ty pattern when undesirable side effects 
are produced by allowing the complete 
set of instantiated class objects to share a 
property (such as when executing within 
a browser environment). 

Structure. The Shared Property pattern 
is a composite of three classes: Shared- 
Properties, PropertySharer, and Property- 
Owner. Figure 4 shows the structure of 
the Shared Property pattern. 

Participants. SharedProperties, (Plane- 
Properties), the attributes to be shared by 
a group of objects; PropertySharer (Plane), 
an object belonging to the group sharing 
the common attributes; PropertyOwner 
(WebCAD), the object responsible for the 
construction and storage of the common 
SharedProperties object. « 

Collaborations. Clients access shared 
properties via a reference to the unique 
SharedProperties object, instead of through 
a static class variable of the sharing class. 
Intermediate classes in the composition 
hierarchy are obliged to pass on that ref- 
erence to the SharedProperties object un- 
til it reaches its ultimate destination in the 
PropertySharer class objects. 

Consequences. The Shared Property 
pattern allows the conventional behavior 
of static class variables to be reclaimed in 
an environment where static variables do 
not have their traditional semantics. 

Implementation. The steps to con- 
vert a class that uses static fields to one 
that implements the Shared Properties pat- 
tern are: 


1. Extracting the shared properties. Extract 
any fields defined as static in the class 
to be converted (the PropertySharer) 
and place them in their own separate 
class (the SharedProperties class). The 
crucial step is to ensure that they are 
defined as regular members in the new 
class rather than as static fields. 

2. Creating the shared properties object. 
Construct a single instance of the 
SharedProperties class as part of the ap- 
plet class’s construction and record it 
in a private instance variable. If the 
values for the initial state of this object 
are not yet known, simply create an ob- 
ject using default values. Being shared, 
changes made to this object are made 
immediately apparent to members of 
the PropertySharer class, so initializing 
the values in the SharedProperties 
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object may be deferred until the point 
that the first PropertySharer object needs 
to use them. 
3. Making the shared properties accessi- 
| ble. The SharedProperties object needs 
| to be available wherever objects of the 
PropertySharer class are instantiated. 
The easiest way to achieve this is to 
pass a reference to the applet’s Shared- 
Properties object down the function call 
| hierarchy through the constructors of 
| any intermediate classes (or, alterna- 
| tively, a reference to the Applet object, 
from which the SharedProperties object 
may be obtained). 

4, Associating the objects with their shared 
properties. Simply provide an additional 
argument to each of the constructors 
for the PropertySharer class. A refer- 
ence to the SharedProperties object 
should be stored in a private instance 
variable. 

5. Implementing Accessors and Mutators. 
Any existing accessor (get methods) or 
mutator (set methods) functions in the 





Listing One 


class Plane { 
static int displayMode; 
static double scaleFactor; 
static double rotation, elevation; 


Listing Two 


class WebCAD extends Applet { 
WebCAD() { 


elevation, /*other args*/); 
} 

} 

class Cube { 


double elevation, /*other args*/) { 


elevation, /*other args*/); 





— 


Listing Three 


class WebCAD extends Applet { 
WebCAD() { 
cubes[@] = new Cube(this, /*other args*/) ; 


} 
j 
class Cube { 
Cube(Applet applet, /*other args*/) { 
topFace = new Plane(applet, /*other args*/); 


} 
} ' 
class Plane { 
Plane(Applet applet, /*other args*/) { 
this.applet = (WebCAD) applet; 


} 
void draw(Graphics g) { 
switch (applet.displayMode) { 
case DM_WIRE_FRAME: 
case DM_SOLID_OBJECT: 
} 
} 
private WebCAD applet; 
} 


| Listing Four 


class PlaneProperties { 
int displayMode; 
double scaleFactor; 

| double rotation; 

double elevation; 
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cubes[@] = new Cube(displayMode, scaleFactor, rotation, 


Cube(int displayMode, double scaleFactor, double rotation, 


topFace = new Plane(displayMode, scaleFactor, rotation, 


PropertySharer class, along with any 
other references to the erstwhile static 
fields, are modified to reference the 
fields in the SharedProperty object by 
prefixing them with the name of the 
SharedProperty instance variable. The 
accessors and mutators remain in this 
class and are not relocated in the 
SharedProperties class, so that use of 
the design pattern is as transparent to 
users of the class as possible. 


Conclusion 

Classes that use static fields with their tra- 
ditional semantics almost always give un- 
desired behavior when instantiated in an 
applet environment. To produce portable 
code or to use static fields with their con- 
ventional meaning in applets, an alterna- 
tive idiom is required. 

We've presented two possible solutions. 
For simple cases— and especially where 
a value is global to the applet and not par- 
ticular to any one class—the approach 
of storing the data directly in the main Ap- 


plet object and passing the applet object 
down to the other helper classes is con- 
cise, effective, and extensible. 

Where there is a deliberately exclusive 
relationship between a set of shared data 
and objects of a particular class, the Shared 
Property pattern is superior because, in 
addition to all the benefits of the first ap- 
proach, it explicitly establishes a tightly 
coupled relationship between the two 
classes. 

We recommend that you use either of 
the solutions we present even when pro- 
ducing classes targeted for stand-alone 
Java applications, so that if you wish to 
utilize those classes in an applet at a lat- 
er time, you may reuse them freely with- 
out having to make any modifications. 

The working Java applets from this ar- 
ticle may be viewed at http://effectivejava 
.com/steve/iapps/ and at http://www.dcda 
.co.nz/jmc/iapps/. 
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PlaneProperties(int displayMode, double scaleFactor, 
double rotation, double elevation) { 


this.displayMode 
this.scaleFactor 


displayMode; 
scaleFactor; 


this.rotation = rotation; 
this.elevation = elevation; 


} 


ad 


class WebCAD extends Applet { 


WebCAD() { 


planeProperties = new PlaneProperties(Plane.DM_WIRE_FRAME, 
1.@, INITIAL_ROTATION, INITIAL_ELEVATION) ; 
cubes[@] = new Cube(planeProperties, /*other args*/); 


} 


Listing Five 


class Cube { 


private PlaneProperties planeProperties; 


Cube(PlaneProperties planeProperties, /*other args*/) { 


topFace = new Plane(planeProperties, /*other args*/); 


Listing Six 


class Plane { 


Plane(PlaneProperties planeProperties, /*other args*/) { 
this.planeProperties = planeProperties; 


} 


void draw(Graphics g) { 


switch (planeProperties.displayMode) { 
case DM_WIRE_FRAME: 
case DM_SOLID_OBJECT: 


} 
} 


private PlaneProperties planeProperties; 
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Analysis of 
Algorithms 
Jon Bentley 


ou’ve just designed a nifty new al- 

gorithm. It’s fast, but exactly how 

fast is it? To help you find out, this 

column presents tools and tech- 
niques for analyzing the performance of 
algorithms. It starts with a simple program 
for an important problem, and improves 
it through several versions. 

I originally presented versions of these 
algorithms in Unix Review magazine (June, 
1997). In the process, I observed huge per- 
formance speedups— a factor of 30,000— 
but didn’t take the time to understand the 
details of the run times. This column ana- 
lyzes those algorithms, with an emphasis 
on methods that real programmers might 
use to understand performance. 


The Problem 

The Traveling Salesman Problem (TSP) 
calls for finding the shortest tour through 
a collection of cities. Figure 1 shows the 
shortest circuit through 14 cities that Abra- 
ham Lincoln visited when he rode the IIli- 
nois Eighth Circuit in 1850. This proto- 
typical network design problem often 
arises in applications such as scheduling 
mechanical devices and factory produc- 
tion lines. 

The fundamental mathematical ab- 
straction for dealing with the TSP is a 
permutation— an ordered sequence of 
objects. For instance, there are a total of 
six permutations of the three letters “a,” 
“band, “C" 

abc acb bac bca cab cba 


In general, there are n/ (“n factorial”) 
permutations of a letters, where 
nl=nx(n—1)x(n—2)x...x3x2x1. This is be- 
cause any one of the 7 elements can go 
in the first position, any one of the re- 
maining n—1 elements can go in the sec- 
ond position, and so on down to the sin- 
gle remaining element that must go in 
the last position. The factorial function 
grows very quickly: 


Jon is a Member of Technical Staff at Lu- 


cent Technologies’s Bell Labs. He can be 
reached at jlb@research. bell-labs.com. 
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10!=3 628,800, 
20!=2.43x10!®, and 
30!I=2.65x10°2. 


A TSP tour is a permutation of the in- 
put cities, implicitly closed by the tour by 
traveling from the last city back to the first. 
The program tspl.c (and a sample input 
file, both available electronically; see “Re- 
source Center,” page 5) deals with 7 cities, 
where 7 is a global variable: int n;. 

The code won't peek inside the repre- 
sentation of cities. Instead, it will access 
distances between cities with the distance 
function 


typedef double Dist; 
Dist d(int i, int ;) 


It will represent the tour in the vector p 
int pIMAXN]: 


where p/0..n-1/ is a permutation of the 
cities in the tour. For instance, a tour that 
goes from city 3 to 1 to 4 then 2 (and fi- 
nally back to 3) would be represented by 
the permutation vector 3 1 4 2. 


Algorithm 1: The Easiest Program 

The first algorithm is easy: It will re- 
cursively generate all ”/ permutations, 
and choose the one with least cost. Be- 
cause it inspects so many tours, we can't 
expect the algorithm to be fast; we'll 
soon analyze it to see exactly how slow 
it is. Assume that the variables are ini- 
tialized with 

for Gi = 0; i < n; i++) 

pli] = i; 
minsum = INF; 


The name solvei denotes that this is the 
first version: 


void solvel() 
{ search1(n); 


The workhorse is the following search 
function. The parameter m tells how 
many elements of the vector p have yet 
to be permuted. It operates top-down 
by leaving p/m..n—1/ fixed while it per- 
mutes p/O..m-—-1/: 





void search1Cint m) 


{ int i; 
if (m == 1) 
check1Q); 

else 


for G = m-1; i >= 0; i--) { 
swap(i, m-1); 
search1(m-1); 
swap(i, m-1); 
} 
} 


The swap function exchanges pairs in the 
vector p. The for loop swaps each element 
in turn to the end, recurs with size m de- 
creased by one, then swaps the element 
back to its original place. This code is short 
but subtle, and is worth serious study be- 
fore proceeding. 

When a complete permutation is fixed, m 
is 1 and search calls the check1 function: 


void check1Q 
{ int i; 
Dist sum = d(pl0], p{n-1); 
for G = 1; i <n; i++) 
sum += d(pli-1], pli); 
save(sum); 


This function computes the cost of the 
tour by summing the n—1 distances be- 
tween adjacent points and adding in the 
distance from the last point to the first. 
The save function records the tour and its 
cost, if it is the best so far: 


void save(Dist sum) 
{ int i; 
if (sum < minsum) { 
minsum = sum, 
for G = 0; i < n; i++) 
minpli] = plil; 


The global variables minp and minsum 
record the best tour and its length. 

How slow is this program? The run 
times listed in Table 1 were generated on 
a 200-MHz Pentium Pro machine. The 
first three times are observed values in 
seconds, and the final four times are 
back-of-the-envelope extrapolations. The 
program is indeed pokey. 
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A little mathematics shows why it is so 
slow. The code investigates all 7/ permu- 
tations, and performs 7 distance calcula- 
tions for each (to calculate the cost of the 
tour). The total number of distance cal- 
culations used by the program is there- 
fore nxnl. 

The analysis is confirmed by looking at 
a profile. Table 2 describes one run of the 
program with n=9. This data confirms the 
analysis: The check1 function is called n/ 
(or 362,880) times. The distance function 
is called nxn/ (or 3,265,920) times. This 





Figure 1: The shortest circuit through 
14 cities that Abraham Lincoln visited 
when he rode the Illinois Eighth 

Circuit in 1850. 







Table 1: Run times of Algorithm 1 on 
a Pentium Pro 200. 


Table 2: Profile of Algorithm 1. 


ee 








Table 3: The distance calculations 
performed by Algorithm 2. 
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profile also shows that the distance func- 
tion is indeed the critical operation in this 
program: The dist function and the sgr and 
sqrt functions it calls account for almost 87 
percent of the run time of the program. 

Exercise 1: Try profiling a C implemen- 
tation of an algorithm. 


Algorithm 2: A Fixed Point 

The next algorithm reduces the run time 
by examining fewer permutations. It fix- 
es the last city in the permutation, and 
starts the search at one lower value: 


void solve2() 
{ search1(n-1); 
} 


This code no longer inspects all 7/ per- 
mutations, but still examines all possible 
tours. Algorithm 2 looks at only (7-1)/ 
permutations, and performs m distance 
calculations at each. The total number of 
distance calculations is nx(m-1)/, or pre- 
cisely n/. 

To verify this, we’ll augment the dist 
function with instrumentation code that 
gives key counts without the overhead of 
profiling: 


Dist dist(int i, int j) 

{ 

#if COUNTDISTS 
distcnt++; 

#endif 





Table 4: Run times of Algorithms 1 
and 2. 





Table 6: A spreadsheet for analyzing Cm. 


When the program turns on COUNT- 
DISTS, it increments distcnt at each call. Cf 
the constant is zero, no additional run-time 
overhead is incurred.) Table 3 shows the 
number of distance calculations performed 
by the function. The first and second 
columns show that the number of distance 
calculations is indeed n/. The third row di- 
vides the current number of distance cal- 
culations by the previous number. In this 
case, those ratios repeat precisely the first 
column, and characterize factorial perfor- 
mance. 

Of course, it is always important to ver- 
ify that theory translates into reality; see 
Table 4. Both algorithms increase their run 
times by about a factor of 7 as 1 increases, 
and the run time of Algorithm 2 on 7 is 
a bit more than Algorithm 1 on 7-1. Both 
of these observations are consistent with 
the predicted values. 


Algorithm 3: Sum Times Not 

The next version of the algorithm reduces 
the overhead of computing the sum for 
each permutation from scratch. Instead, 
the code carries along the partial sum in 
a new parameter to the search3 function. 
It initially calls the function with the par- 
tial sum set to zero: 


void solve3() 
{ search3(n-1, 0); 
} 


It adds in the distance from the top city 
to the city next to it at each recursive call: 


void search3(int m, Dist sum) 
{ int i; 


if (m == 1) 
check3(sum + d(p[0], p{1)); 
else 


for G = m-1; i >= 0; i--) { 
swap(i, m-1); 
search3(m-1, sum + d(p{m-1], p{m])); 
swap(i, m-1); 
} 
} 


The new check3 function adds in the dis- 
tance from the first to the last city: 


void check3(Dist sum) 
{ sum += d(p{0], p{n-1]); 
save(sum); 
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This algorithm still looks at (n—1)/ per- 
mutations, and I guessed that it would av- 
erage roughly one distance calculation for 
each. I therefore expected another 
speedup of about a factor of 7. Table 5 
lists the run times. Where I hoped for a 
speedup of around 10, I observed a 
speedup of just 2 or 3. What gives? 

I enabled the operation counts, and ex- 
amined the sequence c,,, which denotes 
the count of distance functions used when 
the first parameter to search is m. I start- 
ed with small values: 


Cy = 2, cz = 6, cg = 21. 
The sequence begins: 
2, 6, 21, 88, 445, 2676, 18739, ... 


My first approach was to check this se- 
quence in Neil Sloane’s online encyclope- 
dia of integer sequences, but it wasn’t there 
at the time. 

Exercise 2: Try looking up the sequence 
at Sloane’s web site at http:/Avww.research 
.att.com/~njas/sequencey. 

The next step involves finding a re- 
currence relation to describe the se- 
quence in terms of its earlier values. 
When m is one, there are two cities in 
the array and the algorithm makes two 


I subtracted c,, from (m+1)/ and took sev- 
eral ratios, all without insight. I finally 
stumbled across the ratio c,,/m/ shown in 
the fourth column, in which IJ at long last 
recognized a pattern. 

In high school, I memorized the nat- 
ural logarithmic base e to accuracy 
2.718281828459045...(Your guesses about 
my social life in high school may prove 
uncannily accurate.) My complete 
spreadsheet converged to 14 of those 
decimal digits at m=18, where floating- 
point accuracy finally gave out. This led 
immediately to the conjecture that: 


Cm = m!x(1 te) 
But how does the ubiquitous e sneak into 


my TSP program? Subtracting one from 
the right column gives the sequence 1, 2, 


DR. SOESS CD-ROM LIBRARY : 


De Dats 





2 1/2, 2 2/3,..., which I recognized as the 
partial sums of the Taylor series expan- 
sion of e* evaluated at x=1. This suggests 
the theorem: 


Cm=m!X(1+1/0!4+1/1!+...+1/Gm-D) 


Exercise 4: Use mathematical induction 
to prove that this solution satisfies the re- 
currence relation. 





Table 8: Run times of Algorithms 3 
and 4. 
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comparisons (between that pair and from 
the first to the last): c;=2. When there are 
m=2 cities, the algorithm swaps each of 
them to be the last, calls itself recursive- 
ly on size m-—1 (at cost C,,_1), and then 
makes one additional distance calcula- 
tion for each: ¢C,=mx(Cp_1+V). 

I verified the first few values by hand, 
then used a spreadsheet to check larger 
values against the instrumented runs. 

Exercise 3: How would you construct 
that recurrence in a spreadsheet? You 
may want to warm up by constructing 
the sequence ml. 

The recurrence matched the experi- 
ments. To understand the sequence, I 
toyed further with the spreadsheet. Be- 
cause I expected c,, to be similar to fac- 
torials, the third column in Table 6 shows 
m!. 1 next played with several different 
comparisons of the second and third rows: 
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Table 7: Distance calculations used 
by Algorithm 4. 
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Exercise 5: Play further with a spread- 
sheet to discover additional patterns in the 
SEQUENCE Cy. 

Because Algorithm 3 makes the orig- 
inal call search3(n-1, 0), it uses c,_1 or 
approximately (7—-1)/x(1+e) calls to the 
distance function. The speedup is not 
n, but rather roughly n/(1+e), which 
closely matches the experimental run 
times. 


Algorithm 4: Pruning the Search 
“Don’t keep doing what doesn’t work” 
is good advice for programs as well as 
people. As the program progresses, sum 
can never decrease: Adding edges only 
increases the tour length. The program 
can therefore terminate the search when 
sum grows too large: 


2676 

18739 
149920 
1349289 





Table 9: A spreadsheet for Exercise Sy 


End the Chaos... 


void search4(Gint m, Dist sum) 
{ inti: 
if (sum > minsum) 


return; 
if (m == 1) 
... as before ... 


Each previous algorithm runs in the 
same time, for any size n, regardless of 
the particular input data. The run time of 
Algorithm 4 varies dramatically with its in- 
put. Table 7 shows the number of distance 
calculations it uses on one sequence of 
inputs, where each input adds a single 
city of Lincoln’s tour. By n=11, pruned AI- 
gorithm 4 uses about a factor of 10 few- 
er distance calculations than Algorithm 3. 
Furthermore, the ratios of distance calcu- 
lations show that the run time does not 
increase by a factor of n at each stage, 
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but rather by a smaller factor (apparently 
near 5 for this range). As Table 8 shows, 
run-time experiments confirm both pre- 
dictions. 

Algorithm 4 is much faster than Algorithm 
3, and its run time grows more slowly. 

Exercise 6: Analyze Algorithm 4 in greater 
detail across a wider variety of inputs. 


Tools and Techniques 
Programming tools and techniques for an- 
alyzing algorithms include: 


e Profilers. These general tools help us 
identify the key operations in code and 
their associated counts. 

¢ Operation Counts. After identifying dis- 
tance calculations as a critical operation, 
we added instrumentation code to count 
those operations. 

e Integer Sequences. You can analyze 
some algorithms by studying their op- 
eration counts with tools such as ratios 
and Sloane’s online encyclopedia of se- 
quences. 

e Spreadsheets. Spreadsheets provide a 
convenient way for storing and analyz- 
ing program performance data. 

e Mathematics. Detailed analyses can 
sometimes provide real insight into real 
programs. 

e Experiments and Common Sense. Al- 
ways run experiments to ensure that 
your theory matches practice. 


Stay Tuned! 

In my next column, Ill examine how 
code-tuning techniques speed up the var- 
ious algorithms. 


Solutions to Selected Exercises 
Exercise 2. Although the sequence was 
not there when I originally looked it up 
in April, 1998, it has since been added. 
(Amazing how one sequence pops up in 
various contexts!) The encyclopedia now 
gives the recurrence, but not the approx- 
imation derived in this column. 

Exercise 3. To compute m/, first set the 
A column to the integers 1, 2, 3,... Next set 
B1 to 1, then set B2 to =A2xB1, and drag 
it down the B column. To compute c,,,, set 
the first column as before, set B1 to 2, set 
B2 to =A2x(B1+1), and drag it down. 

Exercise 5. The first four columns in 
Table 9 are from the previous spreadsheet. 
The fifth column shows the difference be- 
tween elements in the fourth column, and 
the fifth column is the inverse of the fourth 
column. I was delighted to see the facto- 
rials pop out in the fifth column. This is 
an alternate derivation of the fact that 
Cy, /M!=141/0/4+1/1/+...+1/(n-VD. 
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Joints in Space 


Dennis E. Shasha 


@, reat architects like Ecco. They rec- 
—_ ognize his ability to solve tough 
|," structure and placement problems. 

© The feeling is mutual. Ecco doesn’t 
care much for glass towers or phony Vic- 
torian, but he holds a deep affection for 
Italian piazzas, well-designed airports, and 
clever approaches to traffic flow like the 
Guggenheim museum in New York or the 
Senkai complex in Tokyo. 

The kinds of problems he’s attracted to, 
though, usually have to do with extreme 
environments such as Antarctica or un- 
derwater cities. 

Jordan Tyler brought him such a prob- 
lem one fine winter day. Tyler’s reputa- 
tion had preceded him. A descendant of 
a famous railroad designer, he combined 
mathematics, architecture, and physics in 
all his creations. It was for that reason that 
a consortium of space exploration com- 
panies led by none other than Mr. Nog 
Tugget (readers may remember this un- 
savory descendant of a ruined Alaskan 
gold rusher) had approached Tyler to de- 
sign a modular space station. 

Tyler’s design specified rooms consist- 
ing of symmetric cubes 10 meters on a 
side with a single manhole-sized door in 
the center of each of the six cube faces. 

“We decided there was no reason to 
prefer one direction to another in space,” 





Dennis, a professor of computer science at 
New York University, is the author of The 
Puzzling Adventures of Dr. Ecco (Dover, 
1998), Codes, Puzzles, and Conspiracy 
(W.H. Freeman & Co., 1992), Database 
Tuning: A Principled Approach (Prentice 
Hall, 1992), and (coauthored with Cathy 
Lazere) Out of Their Minds: The Lives and 
Discoveries of 15 Great Computer Scien- 
tists (Springer Verlag, 1998). He can be 
contacted at DrEcco@ddj.com. 
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ECCO'S OMNIREURIST CORNER 


Tyler explained. “In fact, I’d like to get 
away from the idea of rooms altogether, 
but the oldtimers want to be able to quar- 
antine areas in case of leaks or toxicity 
problems. They’ve also asked me to help 
them design the interiors of the 48 cubes, 
but the first problem is how to arrange 
them and which company to place in each 
cube. That’s what I’m coming to you for.” 

“What have they told you about what 
goes on in the space station?” Ecco asked. 

“It is a deep commercial secret, it 
seems,” Tyler answered. “All I know is 
that 15 companies are involved, each of 
which occupies one or more cubes that 
must be connected. I'll indicate this to you 
with a letter for each corporation, because 
their names are also secret.” 


Number of rooms 
10 


Company 


> 


oN) 


O22" ASF 2oO apo 
RR NNR Rr NM NF NY OF 


“Also, certain pairs of companies, due to 
joint ventures, must have neighboring cubes. 
Here, I represent the necessary adjacency 
relationships in alphabetical order. For ex- 
ample, the fourth line says that some room 
of D must be next to some room of A and 
some (possibly different) room of D must 
be next to some room of C, and so on for 
E, F, and G. There is some redundancy in 
this representation, but I hope it is clear. 





“Lack of adjacency imposes no con- 
straint, by the way. The fact that B is not 
in the adjacency set of D, for example, 
means that it is not necessary for any room 
of B to border any room of D, though this 
is allowed and even desirable. These com- 
panies engage in all manner of joint ven- 
tures and those joint ventures may require 
an exchange of materials within the space 
station.” 


A—BCDEFHIJLM 


B—A C 
C—ABDEGM 
D—ACEFG 
E—ACDFGJKLMNO 
F—ADEJN 
H—AIL 
I—AHJL 
J-AEFI 
L—AEHI 
M—ACE 

GC Dik 
K-EGO 

N—E F 

O—EK 


“From your encoding, can we conclude 
M is a card sharp and K is an empath?” 
asked Ecco with a smile. 

“Random letters produce tantalizing pat- 
terns,” Tyler answered with a chuckle. 

“And when you say the 13 rooms of A 
must be connected, all you mean is that 
an astronaut can float from any room in 
company A’s space to any other without 
passing through any other company’s 
cubes?” asked Liane. 

“Correct,” Tyler said, nodding to the 10- 
year old. “As I said, they are all very jeal- 
ous of their secrets, more secretive in 
many ways than countries at war.” 


Reader: Ecco and Liane succeeded in 


discovering a placement of the cubes in 
three dimensions and an assignment of 
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the companies to cubes that satisfied these 
constraints. Can you do it? 


After Tyler left, Ecco turned to me. 

“My dear professor,” he said, “do you 
think it is possible to do better? I mean, 
can you find a design that satisfies these 
constraints but that establishes a proper 
superset of these adjacency relationships? 
If so, what is the design and what is the 
largest number of adjacency relationships 
you can find?” 


Reader: Would you like to give it a try? 


Last Month’s Solution 

When trains can carry only 50 passengers 
and all stations hold only two trains, this 
is the best Ecco and Liane could come 
up with: 


Time Trips Started 

0 min DBC, EAB, ABF, BFE, FEA 

15 min CBD 

30 min GBA, BAE, AEF, EFBD, DBC, CBG 
45 min CBD 

60 min DBC, FBC, EFB, AEF 

75 min CBD 

90 min Some in transit 

105 min Everyone done 


When trains can carry 150 passengers 
and station B can hold four trains, but all 
other stations hold only two trains, Liane 
suggested the following: Put the extra 50 
person train at station C, so C has two 
trains to start with. Further, assume the 50 
person train that used to start at B now 
starts at F. 


Time Trips Started 


0 FBAE (starts with 50 bound to A, 
picks up 100 passengers at B and 
proceeds to E after dropping 
passengers at A), DBC, CBD, 
AEF, EFBD (150 load at E; train 
drops off 100 at B and carries 50 
to D), CBG (50 person train). 


is GBA (50 person train), FBC (50 
person train) 


Traffic at Station B: 

time 0 — 0 

time 15 — one from FA, one from DBC, 
one from CBD, one from CG 


time 30 —- one from EFBD, one from GC, 
one from FC. 


Reader Solutions to the Fair Swedes 
Problem 

Many readers found correct solutions to 
the Gutoldenborg map problem (DDj, Jan- 
uary 1999), most by “sitting at the kitchen 
table and scribbling” as one reader put it. 
Others used genetic algorithms, gradually 
refining the solution. Still others used ex- 
haustive search, but more on that later. Fi- 
nally, Joe Celko managed to find a solu- 
tion in SQL— amazing! These readers sent 
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me solutions: Juan Pancorbo, Kerry D. 
Millen, Pearl Pauling, Ralph Nebiker, Blaine 
Deal, Scot Billman, Michael S. VanVertloo, 
Jimmy Hu, Roger Alley, Michael Goldberg, 
Burghart Hoffrichter, Ernst Munter, Odd 
Tangen, Mike Robinson, John “Goodboy” 
Holland, Serguei Patchkovskii, Friedrich 
von Solms, Franco Venturi, Mark J. Mur- 
phy, Jeff Gerald, James Heginbottom, Bill 
Rooney, Jean-Philippe Langlois, Bryan S. 
McDaniel, Jacco Kulman, Kakulidis Iu- 
lian, Joshua Lynch, Viktor Bresan, Peter 
Hansen, Mathew R. Davies, and Benjamin 
C. Chaffin. 

As Ralph Nebiker wrote, “This could 
be a really difficult problem if the 
boundary conditions and populations 
weren't quite so well defined. So, I de- 
cided to pursue that angle and per- 
suaded some respondents to try to find 
14 districts each having approximately 
12,000 people and to make it so that the 
maximum deviation from 12,000 was 
minimal.” 

The three best solutions to this second 
problem came from Roger Alley, Friedrich 
von Solms, and Serguei Patchkovskii 
(whose exhaustive search method used 
2.5 hours on a cluster of 39 500-MHz 
21164 Alphas). The maximum deviation 
from 12,000 these solutions. produced was 
a surprisingly low 29. 


WIBU-KEY 


Solution 

(11 48 64) 
Population: 11992 
(8 15 19 37 66) 
Population: 11975 
(2 14 36 40 56) 
Population: 12022 
(10 31 46 62) 
Population: 11996 
(12 27 43 57) 
Population: 11996 
(5 25 47 55 58) 
Population: 12008 
(7 24 32 42 61) 
Population: 12005 
(1 23 28 45) 
Population: 12003 
(6 13 30 41 51) 
Population: 11996 
Care AB eps) 
Population: 12005 
(4 9 26 34 49) 
Population: 11971 
(22 38 50 59 65) 
Population: 11987 


Diff: 8 
Diff: 25 
Diff: 22 
Diff: 4 
Diff: 4 
Diff: 8 
Diff: 5 
Diff: 3 
Diff: 4 
Diff: 5 
Diff: 29 
Diff: 13 


(16 20 39 53 54 60) 


Population: 12020 
(18 29 35 44 63) 
Population: 12000 
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PROGRAMMER’S BOOKSHELF 


The General, 
the Particular, 


and the Just Plain Odd 


Gregory V. Wilson and Steve Chartley 


ost of the computer-related books 
on my shelves fall into one of 
three categories: those that de- 
scribe how to do particular things, 
those that discuss why things ought to be 
done a particular way, and those that can 
be summed up as saying, “Gosh, com- 
puters are really cool!” 

Tom Armstrong’s The Active Template 
Library: A Developer’s Guide is a perfect 
example of the first type. The Active Tem- 
plate Library (ATL) is Microsoft’s most re- 
cent attempt to make its COM object mod- 
el more accessible. The library consists of 
a set of C++ macros and classes that en- 
capsulate and partially automate basic 
COM operations. Both in spirit and in im- 
plementation, the ATL is similar to the Mi- 
crosoft Foundation Classes (MFC), which 
serve the same purpose for Windows GUI 
programming. Instead of buttons, sliders, 
and message boxes, however, the ATL en- 
capsulates reference counting, dispatch in- 
terfaces, and threading. 

Armstrong’s book starts with a short de- 
scription of C++ templates, and then de- 
votes a little over 60 pages to a discussion 
of what COM does and how it works. 
These two chapters are succinct, infor- 
mative, and easy to follow. The third chap- 
ter, which introduces the ATL, is harder 
going, primarily because the ATL is a com- 
plex library. Armstrong does his best, and 
provides some helpful diagrams showing 
the derivations and relationships between 
various ATL classes. However, I think that 
the chapter would benefit from more dis- 





Greg is the author of Practical Parallel Pro- 
gramming (MIT Press, 1995) and coedi- 
tor with Paul Lu of Parallel Programming 
Using C++ (MIT Press, 1996). Greg can 
be reached at guwilson@interlog.com. Steve 
is a developer with Visible Decisions Inc. 
and can be contacted at stevec@vdi.com. 
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cussion of the problems that v various ee 
tures are trying to solve, and from a slow- 
er pace in general. 

Each of the remaining eight chapters 
dissects one of the ATL’s key features — 
the use of IDL, containment and aggre- 
gation, automation, and so on. Each chap- 
ter begins by explaining why the particular 
feature exists, then walks the reader 
through an example. (The source for these 
examples is available at the author’s web 
site.) The two appendices discuss native 
support for COM in Microsoft’s Visual C++, 
and the features that COM+ (COM’s suc- 
cessor) is supposed to have. 

Overall, I found The Active Template Li- 
brary very useful: I’m trying to get up to 


A 


Electronic Review of 
Computer Books 





speed with COM for a new project, and 
have been disappointed by how difficult 
most older books make it. However, the 
book was weakened by a lot of irritating 
typographical errors, both in the text 
(“pawwing” instead of “passing,” for ex- 
ample, and some bad line splices), and, 
more importantly, in the code listings. 
While I hope that a second edition ap- 
pears soon to fix these, the book is still 
the most readable technical introduction 
to the ATL I’ve come across. 

James Coplien’s Multi-Paradigm DESIGN 
for C++ (yes, that’s really how the title is 
capitalized) lives on the other tail of the 
bell curve for serious computing books. 
Coplien is best known for his Advanced 
C++, a widely read book on how to do 
complex things in a language which is fair- 
ly complex in its own right. In this new 
book, Coplien moves beyond smart point- 
ers, the letter/envelope- design pattern, and 
the pitfalls of operator overloading to look 
at when and why you should use inheri- 
tance, templatization, function overloading, 
and the various other abstraction mecha- 
nisms that C++ now provides. 

Or at least I think that’s what he’s do- 
ing, but to be honest, I’m not really sure. 
Multi-Paradigm DESIGN for C++ is writ- 
ten at such a high level of abstraction that 
I often couldn’t tell exactly what point the 
author was trying to make. Occasionally, 
I would be able to seize on something 
concrete, such as the parallels between 
templatization, which give compile-time 
polymorphism, and the inheritance of vir- 
tual methods, which give run-time poly- 
morphism. Most of the time, though, I felt 
like I was in the presence of someone 
who'd had a glimpse of something fun- 
damental, but whose every attempt to put 
it into words ended up sounding banal. 
As depressing as it may be, I think ’m 
simply not smart enough, or experienced 
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enough, to understand what Coplien is 
trying to say. 

On the bright side, I’m pretty sure I un- 
derstood Carlton Egremont II’s Mr. Bun- 
ny’s Guide to ActiveX. This book belongs 
in the “computers are cool” category— or 
more exactly, in the “really geeky humor” 
category. For example: 


The familiar dot “.” symbol from Internet ad- 
dresses is used in this book to terminate sen- 
tences. 


Well, okay, it’s funnier in the book. So 
is the side view of a dialog, and (my fa- 
vorite) the Visual Basic 5.0 splash screen 
on page 40, which looks suspiciously like 
a memory-access violation message box. 
Yes, the book runs out of steam about 
half-way through, but it’s still very funny, 
and let’s face it, where else are you go- 
ing to read something like: 


..you form windows using forms. A form is 
a window that you form. At first forms are un- 
formed. You must form your forms using the 
former designer (formerly the former). In the 
form former, an unformed form forms a uni- 
form formation of dots.... 


—G.V.W. 


here is a quiet revolution under way 

in how we design object-oriented 

software. Reusable object patterns, 

first popularized in the book Design 
Patterns, by Erich Gamma et al. (Addison- 
Wesley, 1995) are changing the way we 
think about, talk about, and design object- 
oriented systems. Design patterns describe 
recurring problems and their core solu- 
tions in such a way that the solutions are 
reusable in different problem contexts. 
Design Patterns (also called the Gang-of- 
Four book, or just GoF) has become re- 
quired reading for object-oriented de- 
signers, and has spawned its own virtual 
community, complete with an annual con- 
ference called PLoP attended by OO 
heavyweights, including the likes of Grady 
Booch, Frank Buschmann, and the GoF 
authors themselves. 

All this attention has made GoF the 
benchmark against which other pattern- 
oriented publications are judged. GoF, it- 
self, has been criticized for being difficult 
to understand. Since design patterns in- 
corporate both the problem and the so- 
lution, you really need an understanding 
of their problem context for them to make 
much sense. Good examples would real- 
ly be helpful here, but GoF is a bit thin 
on examples. The examples it does pro- 
vide are snippets of either C++ or 
Smalltalk, and neither of these is of much 
help to Java programmers. 

This is where Patterns in Java, by Mark 
Grand, really fills a void. Grand adopts the 
format of GoF and restates its patterns us- 
ing Java. PiJ organizes 41 design patterns 
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into six categories. (In contrast, GoF con- 
tains 23 patterns grouped into three cate- 
gories.) The extra patterns described by 
Grand are representative of the many ad- 
vances that have occurred in the three years 
since GoF was first published. Grand also 
presents a group of five “fundamental” de- 
sign patterns. These are object-oriented 
techniques described in the subtext of GoF, 
but not explicitly presented as patterns. 


Patterns In Java 
organizes 41 design 
patterns into six 
categories 


Grand also seizes an opportunity to de- 
scribe seven concurrency patterns that are 
conspicuously missing in GoF and ac- 
knowledged as such by its authors. 

As you begin reading PiJ, you get the 
sense that one of Grand’s goals is to make 
patterns more accessible to the everyday 
OO programmer. In Chapter 1, he pre- 
sents a simple introduction to patterns, 
where he describes the nine topics that 
form the body of each pattern. This is a 
welcome simplification of the 12-topic 
structure used in GoF. In Chapter 2, Grand 
presents an overview of the Unified Mod- 
elling Language (UML) that is used for di- 
agramming in PiJ, While UML is richer and 
more expressive than the notation used 
in GoF, I must confess that I prefer the 
simplicity of the GoF notation (for the de- 
scription of patterns, at least). The cardi- 
nality, type specification, and scoping ex- 
pressed in Pi/’s UML diagrams sometimes 
convey more detail than is necessary and 
obscure the simplicity of the patterns. 

In Chapter 3, Grand makes a concession 
to methodology, by presenting an overview 
of the software development process. This 
is essentially a 10-page introduction to UML 
“use cases” and the requirements for spec- 
ification, analysis, and design, which form 
the framework of every software develop- 
ment project. You’ve probably read this 
kind of thing before, but the review still 
gets you in the mood for what follows. 

Each of the remaining six chapters in 
PiJ describes a collection of related pat- 





terns. Chapter 4, “Fundamental Design Pat- 
terns,” describes five basic patterns that 
form the building blocks for other pat- 
terns. Chapter 5, “Creational Patterns,” de- 
scribes six patterns for implementing dy- 
namic, decision-driven object creation 
mechanisms. Chapter 6, “Partitioning Pat- 
terns,” describes three patterns that pro- 
vide guidance in how to partition com- 
plex functionality into multiple, loosely 
coupled objects. Chapter 7, “Structural Pat- 
terns,” presents nine patterns for manag- 
ing complex arrangements of objects at 
run time. Chapter 8, “Behavioral Patterns,” 
describes 11 patterns that organize and 
combine the behavior of objects to pro- 
mote reuse. Chapter 9, “Concurrency Pat- 
terns,” presents seven patterns for coor- 
dinating concurrent operations, like 
controlling access to shared resources, and 
ordering the sequence of execution. 
There is considerable debate within 
the pattern community as to what con- 
stitutes a proper definition of “design pat- 
tern.” GoF’s stature within the pattern 
community clearly makes it the reference 
definition for design pattern. But with 
each new pattern publication, the defi- 
nition of design pattern seems to broad- 
en. For example, some will argue that 
Grand’s interface pattern is not a pattern 
at all, but is instead simply a feature of 
the Java language. In my opinion, this is 
a red herring issue. The real benefits that 
PiJ, GoF, and other pattern catalogs of- 
fer is that they convey knowledge about 
object-oriented problems and solutions. 
By defining a standard format, these pub- 
lications make this knowledge widely ac- 
cessible and reusable, which has long 
been the Holy Grail of object-oriented 
architecture. So, whether Pi/’s patterns 
conform to GoF’s definition of design 
pattern, by applying Pi/s techniques, 
your Java projects will benefit from im- 
provements in scalability, reusability, and 
maintainability. PiJ is well written and 
reasonably easy to understand. It’s also 
one of the few pattern publications that 
is truly Java centric. If for no other rea- 
son than this, Patterns in Java deserves 
two thumbs up. 
—S.C. 
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| —T , Ms Cand C++ DOCUMENTATION TOOLS (v. 7.0) 
_.° C-CALL ($69) Graphic-tree of caller/called function hierarchy, cross-reference, 


1 file/function index. 
S Oo FTWA R E P R O G R A M S fy © C-CMT ($69) Creates/inserts/updates comment-blocks (functions/identifiers 
LOQking for partners in profit! 


used) for each function. 
e C-METRIC ($59) Calculates path complexity, counts lines with comments, code, 
. You provide the go Small Business, SOHO, Financial, 
Personal Productivity or “how-to” software program. E-Z Legal 


'C' statements. 
e C-LIST ($69) Lists and action-diagrams, or reformats source into user-selected 
Software will provide packaging, duplication, marketing and 
Sales personnel to generate royalty checks for you. 


standard formats. 
e C-REF ($69) Creates cross-reference of local/global/define/parameter identifiers. 
With distribution in more than 6500 retail outlets, we have 
the experience to launch, market and sell your program at retail 


e C-DOC ($199) PACKAGE All 5 programs integrated as DOS program. <10,000 
|) 21.10 cost to you! Dennis Liptrot: 1-800-822-4566 


lines. C-BROWSE Windows graphic-tree viewer. 
e C-DOC Professional ($299) DOS, Windows, 0S/2, 1,000,000+ lines. 
384 S. Military Trail, Deerfield Beach, FL 33442 
Phone (954) 480-8933 ¢ Fax (954) 480-8906 


e NEW VER 7.0! WEB HTML REPORTS! 
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The most recent version of NobleNet’s 
Nouveau Object Request Broker (ORB) 
provides direct COM, CORBA, RPC, and 
Java support, without bridges or wrap- 
pers. Nouveau was also designed to work 
directly with firewalls. The latest version 
of Nouveau features Portable Object 
Adapter (POA) support, Naming Services, 
embedded CORBA, and a C mapping. 
Nouveau runs on Windows NT/95/98, So- 
laris, ALIX, SCO-UNIX, and HP-UX. 
NobleNet Inc. 

337 Turnpike Road 

Southboro, MA 01772 

508-460-8222 

http://www.noblenet.com/ 


SourceOffSite Professional Edition Version 
1.1 is a remote access tool designed for com- 
panies with remote development teams that 
need fast and secure access to a centralized 
SourceSafe database via a TCP/IP connec- 
tion. Version 1.1 features data compression, 
additional project level features such as Pro- 
ject Checkout and Checkin, and a 56-bit en- 
cryption build for international users. 
SourceOffSite 

6 Dunlap Court 

Savoy, IL 61874 

217-356-3213 


http://www.sourceoffsite.com/ 


Topanga Software has announced Topan- 
ga SchematicMaker, a tool for creating elec- 
trical schematics, flowcharts, diagrams, fa- 
cility maps, and tables. The Technical 
Edition costs $79.99, and the Business Edi- 
tion costs $49.99. 

Topanga Software Corp. 

200 Suburban Road, Suite Al 

San Luis Obispo, CA 93401 
805-546-8088 

http://www .topangasoftware.com/ 


InstantObjects has introduced Instant- 
Objects, a web application development 
platform for developing, deploying, and 
maintaining dynamic, e-business web sites. 
The InstantObjects platform features In- 
stantModeler, a GUI for managing the dy- 
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namic elements of the site; InstantDatabase, 
which manages the insertion, real-time in- 
dexing, and rapid extraction of data; In- 
stantCommerce, a GUI for specifying bill- 
able items and events; InstantExtractor, 
which parses structured or unstructured data 
and moves it to InstantDatabase; In- 
stantServer, a Java web application server; 
and InstantSite, a library of sample user in- 
terfaces. The InstantServer operates on 
UNIX, Windows NT, and MacOs. 
InstantObjects Inc. 

651 Brannan Street 

San Francisco, CA 94107 

415-284-5300 

http://www. instantobjects.com/ 


Micro Focus has announced Net Express 
3.0, a development environment that aids 
the rehosting of Cobol applications to the 
World Wide Web. Net Express 3.0 includes 
an Internet Application Wizard, that builds 
Internet client/server applications from a 
database schema, HTML forms, or exist- 
ing Cobol applications; Form Designer, an 
HTML and DHTML forms editor; CORBA 
Wizard for Orbix, which generates a Cobol 
client and wrapper for a CORBA object; 
and remote debugging. Net Express 3.0 
sells for $3650.00. 

Micro Focus 

701 East Middlefield Road 

Mountain View, CA 94043 

650-938-3700 


http://www.microfocus.com/ 


Raima has released Velocis Database Serv- 
er 2.1, which includes new interfaces for 
multiple development environments, in- 
cluding Rogue Wave’s DBTools.h++, Perl, 
Java, and Delphi. Other changes include an 
integrated repair tool called “dbrepair,” SQL 
functions that support scrollable cursors, 
and an ANSI SQL extension that supports 
development of sophisticated multilingual 
or “Soundex” features. Velocis Database 
Server is available on Windows NT/95, AIX, 
BSDI, HP-UX, Linux, SCO, and others. 
Raima Corp. 

701 Fifth Avenue 

Seattle, WA 98104 

206-515-9477 

http://www.raima.com/ 


Digital Delivery announced Confidential 
Courier 2.1, a turnkey software system for 
controlling who accesses which files and 
at what times. Version 2.1 allows compa- 
nies to distribute the same set of secured 
files, contained in an encrypted Courier- 
PAK, to different recipients while control- 
ling which recipient can unlock which file 
or file set. Version 2.1 features two new 
optional add-ons— CourierPAK App- 
Aware and CourierPAK Access Period — 
that let users control how and when se- 


cured files are accessed. The CourierPAK 
AppAware option permits the author of a 
document to designate which application 
recipients must use to open the Courier- 
PAK. The CourierPAK Access Period op- 
tion allows companies to further safeguard 
confidential information by specifying a 
time window in which files can be viewed. 
Confidential Courier 2.1 is available on 
Windows 3.1/95/98/NT. The basic software 
product is priced at $2995.00, with the 
CourierPAK AppAware and CourierPAK 
Access Period options each costing 
$495.00. The complete file security and 
management package, including all op- 
tions, and key server, is priced at $8980.00. 
Digital Delivery Inc. 

54 Middlesex Turnpike 

Bedford, MA 01730 

781-275-3830 
http://www.digitaldelivery.com/ 


Flashline.com, an online retailer of Java- 
Beans and other software components, has 
announced its global Developer Database 
Program. Flashline’s Developer Database 
Program matches developers to companies 
with project proposals. You can enroll in 
the program for free at Flashline’s web site. 
Flashline.com 

1300 East 9th Street, Suite 1310 
Cleveland, Ohio 44114 

216-861-4000 

http://www .flashline.com/ 


Baltimore Technologies has released Ver- 
sion 2.4 of its UniCERT Certification Au- 
thority (CA) system. The version works with 
a wider range of systems, has an enhanced 
policy management system, and supports 
new standards and regulations, including 
PKIX-1 and various national regulatory re- 
quirements. Version 2.4 adds support for 
Automated RA systems built using Ballti- 
more PKI-Plus, which enable enterprises to 
build certificate enrollment systems where 
authentication is provided by another IT 
system rather than a human operator. 
Baltimore Inc. 

101 East Park Boulevard, Suite 600 
Plano, TX 75074 

972-516-3744 
http://www.baltimoreinc.com/ 


Cognet announced the latest version of its 
proprietary software distribution tool, Cognet 
3.0, which provides an end-to-end solution 
for transporting, packaging, customizing, 
and delivering software applications. Cognet 
features a refined “delta” technology, intel- 
ligent uninstall, and automatic installation to 
a fresh copy of Windows. New features in- 
clude a three-tier client/server architecture, 
multilevel security, a keystroke recorder, and 
Profile and Distribute NT System Services. 
Cognet works with Windows 95/98/NT, and 
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directly supports NT Server as well as Net- 
Ware 3.x, 4.x, and 5.x. 

Cognet Corp. 

465 Columbus Avenue 

Valhalla, NY 10595 

914-747-0770 

http://www.cognet.com/ 


The Stingray division of Rogue Wave Soft- 
ware has announced its Objective Toolkit 
for WFC 2.0, which includes support for 
advanced docking windows and toolbars. 
Objective Toolkit’s docking windows sup- 
port creates a framework that makes any 
component dockable and any area a dock- 
ing target. The docking toolbar support 
builds on the docking window support, al- 
lowing toolbars to be docked along any side 
of the application windows or float as a sep- 
arate window. Objective Toolkit for WFC 
2.0 also features several additional control 
enhancements, all integrated with Microsoft's 
Visual J++ development environment. Ad- 
ditional features include Border Layout, Split 
Layout, Listbox Edit, and Scrolling View. 
Objective Toolkit for WFC 2.0 sells for 
$495.00, and includes full source code. 
Rogue Wave Software Inc. 

5500 Flatiron Parkway 

Boulder, CO 80301 

303-473-9118 
http://www.roguewave.com/ 


ObjecTime Limited has announced Devel- 
oper TestScope, which adds testing and 
debugging automation capabilities to the 
ObjecTime family of tools for the real-time 
software developer. Developer TestScope 
works by first transforming software com- 
ponent behavior requirements into sets of 
test cases. It then generates a test frame- 
work and executes the tests. The results of 
each test are presented in a graphical form 
showing any differences between the re- 
quired behavior and the actual behavior. 
ObjecTime Limited 

340 March Road 

Kanata, ON 

Canada K2K 2E4 

613-591-3535 


http://www.objectime.com/ 


PerlDirect from ActiveState Tool is a Perl 
support program designed for corporate 
IT managers. PerlDirect features validat- 
ed, quality-assured releases of Perl and its 
popular extensions, advice and support, 
a Y2K test suite, incident-based support 
through the Perl Clinic, and a Perl Alert 
weekly bulletin. Basic annual subscription 
rates start at $12,000.00. 

ActiveState Tool Corp. 

P.O. Box 2870 Main Station 

Vancouver, BC 

Canada VOB 3X4 
http://www.activestate.com/ 
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JLOOX 1.0 from LOOX Software is a dy- 
namic graphics and data visualization de- 
velopment tool for Java programmers. 
JLOOX lets Java programmers create high- 
performance user interfaces based on the 
Java2D rendering API. By providing a set 
of high-level graphics components built 
on the Java2D API, JLOOX lets develop- 
ers create high-end graphical interfaces 
that go beyond the capabilities of Swing. 
JLOOX includes JLOOXMaker, an inter- 
active editor for creating complex static 
or animated graphical control objects. 
JLOOX 1.0 sells for $2500.00 for a single- 
user development license, and has no run- 
time fees or royalties. 

LOOX Software Inc. 

4962 El Camino Real, Suite 206 

Los Altos, CA 94022 

650-903 -0942 


http://www.loox.com/ 


NeoLite from NeoWorx is a compressor for 
Win32 executables. Compressed programs 
execute directly, with no separate decom- 
pression step. Compressed candidates in- 
clude EXE, DLL, OCX, and ActiveX files. 
NeoLite sells for $128.00, and is royalty free. 
NeoWorx Inc. 

P.O. Box 969 

Flat Rock, NC 28731 

828-697-7901 

http://www.neoworx.com/ 


Mib Software has announced a freely avail- 
able, online reference to popular, reusable, 
open source and public domain software 
functions, libraries, and applications. The 
comprehensive index, located at http:// 
www .imibsoftware.com/reuse/, links di- 
rectly to home and archive sites where 
source code is available. 

Mib Software 

RR 4 Box 4110 

Saylorsburg, PA 18353 

717-992-8824 
http://www.mibsoftware.com/reuse/ 


TeamShare’s release TeamTrack Version 3.0 
is the most recent version of the company’s 
problem-tracking system for software- 
development teams. Using TeamTrack, soft- 
ware teams can track and prioritize defects, 
customer requirements, change requests, 
and other issues that arise during software- 
development projects, all over the Web. New 
features include Folders, Version Control In- 
tegration, Threaded Notes, and Remote Ad- 
ministration. TeamTrack Version 3.0 costs 
$499.00 for a single-user license, with vol- 
ume discounts available. 

TeamShare Inc. 

1975 Research Parkway, Suite 105 
Colorado Springs, CO 80920 
719-599-4444 
http://www.teamshare.com/ 





The Barcode Suite from SkyLine Tools 
Imaging is a barcode recognition toolkit 
for Delphi, Visual Basic, and C++. Using 
a fuzzy logic algorithm for image recog- 
nition, Barcode Suite identifies and de- 
codes most of the widely used barcode 
standards. Barcode recognition is done 
from a black and white 1-bit image. The 
Barcode Suite currently comes as a DLL, 
and costs $1999.00. : 

SkyLine Tools Imaging 

20537 Dumont Street, Suite A 

Woodland Hills, CA 91364 

818-346-4200 
http://www.skylinetools.com/ 


Lassalle Technologies released Version 2.1 
of AddFlow, its flowcharting and dia- 
gramming ActiveX control. This new re- 
lease now features complete printing and 
previewing capabilities. The AddFlow con- 
trol allows the creation of diagrams, whose 
objects — links and nodes— are fully cus- 
tomizable. An AddFlow diagram can be 
used to display application data and for 
knowledge navigation. Drawings may be 
done interactively or programmatically. 
AddFlow costs $299.00. 

Lassalle Technologies 

247, Avenue du Marechal Juin 

92100 Boulogne 

France 

33 1 46 03 42 20 

http://www .lassalle.com/ 


KL Group Inc. released JClass SwingSuite, 
a suite of JavaBean components for build- 
ing graphic user interfaces. JClass Swing- 
Suite provides a set of extensions and en- 
hancements for Swing in JDK 1.2. Features 
include MDI to manage multiple windows 
inside applications and sophisticated Wiz- 
ards to guide users through complicated 
tasks. JClass SwingSuite includes over 20 
thread-friendly, lightweight components 
for adding features such as proper resiz- 
ing behavior to applications, an advanced 
multicolumn outliner, progress manager, 
enhanced tree views, and spin boxes. 
KL Group Inc. also released the 2.0 ver- 
sion if JProbe, its Java profiling tool. This 
release includes new heap analysis tools 
for finding and eliminating memory leaks. 
JProbe works with the Solaris and Win- 
dows versions of the Java Development 
Kit 1.1 and 1.2. 
KL Group Inc. 
260 King Street East 
Toronto, ON 
Canada M5A 1K3 
416-594-1026 
http://www.klgroup.com/ 
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SWAINE’S FLAMES 


Death Rattle 


ou have to be taken to Foo Bar the first time you go; you’d never find it on your own. 
Yiccs away at the end of what looks like a fire trail amid dense undergrowth beneath a 

stand of redwoods some 1800 feet above Silicon Valley, it can’t be seen from any vantage 
point and has no view. That’s how we like it, those of us who hang out here late evenings to 
argue the burning issues of the day, most of us journalists, most of us cynics. 

I work here one or two nights a week as a relief bartender. On this particular moonless night, 
it was as dark as the grave outside, and just as cold. If the temperature dropped another eight 
degrees, that cold drizzle could actually turn to snow. Customers were huddled over their drinks 
as though to draw heat from them, and conversation was desultory. Then Joe “Curly Joe” Weaver, 
a sort of marginal journalist who writes for inflight magazines and drinks cream soda, brought up 
the Intel boycott. 

“It’s just the coolest thing!” he gushed. “Privacy advocacy groups are calling for a total boycott 
of Intel products, until Intel changes its position on putting digital IDs in its chips. You know 
those IDs could be used to track people wherever they go on the Internet. It’s Big Brother! But 
the people aren’t going to stand for it!” 

Joe is, denizens of Foo Bar generally agree, insufficiently cynical. But there are cynics and there 
are cynics. British journalist Laurence “Larry” Wilde manages to express disdain for practically 
everything with that typically British understated pretension that we Americans mistake for dry wit. 

‘I’m dry, Michael,” Larry said, raising a witty eyebrow and glancing at his glass. I filled it with a 
chardonnay that left it only slightly less dry. “Granted their cause is just,” he said, turning to Joe, 
“their method strikes me as more than a touch quixotic. A boycott of Intel products? So I don’t 
use my laptop today; how does that hurt Intel?” 

Maureen “Mo” McBean is a cynic in the classic American reporter mold. “Privacy” she said, 
pausing for dramatic effect and to finish off her Haig & Haig, “is dead.” She banged the glass down 
on the bar. 

Larry tilted his head to take her in. “It certainly is making a lot of noise.” 

“That’s death rattle,” she said, pushing her glass toward me for a refill. 

“Death rattle? A typically charming Americanism, but its meaning eludes me.” 

Joe jumped in. “Has to do with snakes, doesn’t it?” 

“No, no,” Mo murmured. “It’s a noise made by a corpse. Sometimes things are louder after they 
die than before. Like nationalism and Biblical literalism.” 

“Technically,” I said, jumping in before any nationalists or Biblical literalists who might be in 
the bar could take offense, “death rattle occurs before or at the point of death, not after.” 

“Whatever. The point is, privacy is as dead as a mackerel.” 

“Or as dead as Prodigy Classic?” Joe offered. “Prodigy just shut down its Classic service. Blamed it 
on Y2K problems.” 

“More death rattle,” Mo said. “Prodigy Classic has been dead a long time.” 

Larry pinched his nose thoughtfully. “Actually, ‘dead as a mackerel’ may be dead as a 
metaphor. Some Japanese chap has come up with a way to ship fish alive, using acupuncture. 
Poke them with a needle and the mackerel or salmon catch a nap on their way to the sushi bar, 
where they arrive as fresh as if just caught, although, one imagines, a bit more relaxed. 
Interesting idea, though, that Y2K thing. One could hold seminars on it: ‘Blaming Y2K as a 
corporate public relations strategy.” 

Joe cut in. “There’s a bigger threat to the Net than Y2K, you know.” 

“Marc Andreessen becoming chief technology officer of AOL?” Larry guessed. 

Joe shook his head. “No, States trying to regulate it. Predictions are there’ll be over a thousand 
new Net laws proposed this year in State legislatures.” 

Mo drawled, “Someone should tell them about the concept of interstate commerce.” 

“Well,” Larry said, “if China and Germany and the United States think they can regulate Net 
content, why not the States? It’s a difference in degree, not in kind. Wrong, but not innovatively so.” 

“Politicians. They’re all dead from the neck up,” Mo said. 

Larry smiled. “Ah, but beware their death rattle.” 


Malad Sacdee 


Michael Swaine 
editor-at-large 
mswaine@swaine.com 
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Loaded with 
features... 


Multi - Language 
CodeBase 6.4 works with C, C++, Visual Basic, Java, Delphi, 
ODBC and OLE DB. 





Incredible Speed 

Query a million records in 0.49 seconds. Append 10,000 
records in just 0.65 seconds! All this speed in a super- 
compact library that uses very little memory. 


Royalty Free 


Includes standalone, client and server. 








xBASE Compatible 


Multi-user file compatible with FoxPro, dBASE and Clipper 
data, index and memo files. 





Client/Server 

Advanced security features. New easy-to-use tools for 
monitoring database activity and managing user access. 
Incredibly easy installation and setup! 








Portable 
Runs under Windows 98, 95, NT, CE, 3.1, DOS, Mac, 0S/2, Solaris, 
SunOS, AIX, SCO, Linux, UnixWare, DEC, Alpha, BSDI, HP/UX... 








Enterprise-Level Power 
Includes transaction processing, logging, backup and recovery 
tools, and intelligent queries. 





ay me, And More... 

Recon at Seth Pogue & Data-Aware controls, full-featured report writer and free 
oe technical support. Add-ons available for OLE DB, ODBC and 

Delphi BDE plug-in replacement. 





FREE 30 Day Test Drive 


Test drive the new CodeBase 6.4 for 30 days 
with your own code. No risk. No obligation. 
No royalties. Order today! 


nadeBase Call: 780-437-2410 


SEQUITER|| 


SOFTWARE INC. 


“CodeBase gives ACT! the fast database access 
that contact management users need.” 
- Michael Plasterer, Director of Development, Symantec 
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Award-Winning performance 5 years in a row! 
Fax: 780-436-2999 Email: info@sequiter.com 


© 1999 Sequiter Software Inc. All rights reserved. CodeBase and Sequiter are registered trademarks of Sequiter Software Inc. P.O. Box 783, Greenland, NH 
All other product names are trademarks of their respective companies. In Europe call (44) 181-316-5001 
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